As　a　detection　tool　to　identify　metal　or　alloy,　metallographic　quantitative　analysis　has　re-ceived　increasing　attention　for　its　ability　to　evaluate　quality　control　and　reveal　mechanical　properties.　The　detection　procedure　is　mainly　operated　manually　to　locate　and　characterize　the　constitution　in　metallographic　images.　The　automatic　detection　is　still　a　challenge　even　with　the　emergence　of　several　excellent　models.　Benefiting　from　the　development　of　deep　learning,　with　regard　to　two　different　metallurgical　structural　steel　image　datasets,　we　propose　two　attention-aware　deep　neural　networks,　Modified　Attention　U-Net　(MAUNet)　and　Self-adaptive　Attention-aware　Soft　Anchor-Point　Detector　(SASAPD),　to　identify　structures　and　evaluate　their　performance.　Specifically,　in　the　case　of　analyzing　single-phase　metallographic　image,　MAUNet　investigates　the　difference　between　low-frequency　and　high-frequency　and　prevents　duplication　of　low-resolution　information　in　skip　connection　used　in　an　U-Net　like　structure,　and　incorporates　spatial-channel　attention　module　with　the　decoder　to　enhance　interpretability　of　features.　In　the　case　of　analyzing　multi-phase　metallographic　image,　SASAPD　explores　and　ranks　the　importance　of　anchor　points,　forming　soft-weighted　samples　in　subsequent　loss　design,　and　self-adaptively　evaluates　the　contributions　of　attention-aware　pyramid　features　to　assist　in　detecting　elements　in　different　sizes.　Extensive　experiments　on　the　above　two　datasets　demonstrate　the　superiority　and　effectiveness　of　our　two　deep　neural　networks　compared　to　state-of-the-art　models　on　different　metrics.　©　2020　by　the　authors.　Licensee　MDPI,　Basel,　Switzerland.