Deep-learning　based　image　quality　assessment　(IQA)　algorithms　usually　use　the　transfer　learning　method　that　transfers　a　pre-trained　network　for　classification　task　to　handle　IQA　task.　Although　it　can　overcome　the　problem　of　having　insufficient　IQA　databases　to　some　extent,　it　cannot　distinguish　between　the　important　and　unimportant　deep　features　for　the　IQA　task,　which　potentially　leads　to　inaccurate　prediction　performance.　In　this　paper,　we　propose　a　no-reference　IQA　method　based　on　modelling　of　deep　feature　importance.　A　SE-VGG　network　is　developed　by　using　adaptive　transfer　learning　method.　It　can　suppress　the　features　of　local　parts　of　salient　objects　of　images　that　are　not　important　to　the　IQA　task,　and　emphasize　the　features　of　image　distortion　and　salient　objects　that　are　important　to　IQA　task.　Moreover,　the　structure　of　the　SE-VGG　is　investigated　to　improve　the　accuracy　of　the　image　quality　assessment　on　a　small　IQA　database.　Experiments　are　conducted　to　evaluate　the　performance　of　the　proposed　method　on　various　databases,　including　the　LIVE,　TID2013,　CSIQ,　LIVE　multiply　distorted　and　LIVE　challenge.　The　results　show　the　proposed　method　significantly　outperforms　the　state-of-the-art　methods.　In　addition,　our　method　demonstrates　a　strong　generalization　ability.　©　2020　Elsevier　B.V.