Highlight　detection　is　a　fundamental　step　in　semantics　based　video　retrieval　and　personalized　sports　video　browsing.　In　this　paper,　an　effective　hidden　Markov　models　(HMMs)　based　soccer　video　event　detection　method　based　on　a　hierarchical　video　analysis　framework　is　proposed.　Soccer　video　shots　are　classified　into　four　coarse　mid-level　semantics:　global,　median,　close-up　and　audience.　Global　and　local　motion　information　is　utilized　for　the　refinement　of　coarse　mid-level　semantics.　Sequential　soccer　video　is　segmented　into　event　clips.　Both　the　temporal　transitions　of　the　mid-level　semantics　and　the　overall　features　of　an　event　clip　are　fused　using　HMMs　to　determine　the　type　of　event.　Highlight　detection　performance　of　dynamic　Bayesian　networks　(DBN),　conditional　random　fields　(CRF)　and　the　proposed　HMM　based　approach　are　compared.　The　average　F-score　of　our　highlights　(including　goal,　shoot,　foul　and　placed　kick)　detection　approach　is　82.92%,　which　outperforms　that　of　DBN　and　CRF　by　9.85%　and　11.12%　respectively.　The　effects　of　number　of　hidden　states,　overall　features,　and　the　refinement　of　mid-level　semantics　on　the　event　detection　performance　are　also　discussed.