Scene　classification　is　one　of　the　hot　research　topics　in　the　field　of　computer　vision,　it　is　the　basis　of　the　organization　and　access　for　a　variety　of　image　database,　so　it　has　important　practical　significance.　In　our　previous　work,　we　put　forward　a　novel　fast　scene　classification　method　via　DCT　based　on　the　energy　concentration　and　multi-resolution　characteristics　of　DCT　coefficients.　This　paper　improved　our　previous　work　proposed　a　scene　classification　method　based　on　DCT　domain　using　difference　vectors.　First　of　all,　divided　the　whole　image　into　the　regular　grid　without　repetition,　in　each　grid,　do　DCT　transform　with　the　size　of　8　*　8　get　the　DCT　coefficients　matrix,　extract　the　AC　coefficients　in　the　matrix　get　the　original　vectors;　Then,　selected　N　images　from　each　category　in　the　database　randomly,　calculate　the　average　vector　of　their　original　vectors,　using　the　original　vectors　of　all　images　corresponding　category　subtract　the　average　vector　get　the　difference　vectors　as　the　feature　vectors;　Finally,　based　on　these　feature　vectors　defined　above,　train　classifiers　with　one-vs.-all　support　vector　machine　(SVM).　In　order　to　verify　the　robustness　of　the　proposed　algorithm,　this　paper　has　built　an　image　database　contains　eight　scene　categories　according　to　the　OT　database,　this　paper　conducted　cross　validation　experiment　for　the　proposed　method　in　the　two　databases.　Experimental　results　show　that　the　proposed　method　has　higher　accuracy　and　speed　in　image　classification,　and　has　good　robustness.