The　clustering　algorithm　based　on　mutual　K-nearest　neighbors　(MKNN)　is　presented　to　identify　coherent　structures　in　complicated　fluid　flows,　in　order　to　analyze　the　mass　mixing　and　transport.　First,　both　trajectory　similarity　and　spatial　proximity　are　used　to　describe　and　measure　the　coherence　between　particles.　These　two　identification　criteria　are　frame-invariant　since　they　are　derived　from　the　relative　distances　of　particles.　Then,　the　concept　of　mutual　K-nearest　neighbors　is　introduced　further,　and　particles　with　the　same　cluster　label　are　identified　as　coherent　structures　after　the　initialization　and　merging　process　of　clusters,　while　incoherent　regions　consist　of　incoherent　particles,　which　cannot　form　a　mutual　K-nearest　neighbors　relationship　with　other　particles.　Finally,　the　MKNN-based　clustering　algorithm　is　applied　to　three　examples,　realizing　the　identification　and　tracking　of　coherent　structures.　The　identification　results　show　that　the　MKNN-based　clustering　algorithm　is　robust　to　parameter　K,　and　a　higher　threshold　lambda　of　cluster　quantity　will　be　helpful　to　identify　the　finer　structures　in　flows.　Moreover,　spatial　proximity　performs　better　in　vortex　identification,　and　trajectory　similarity　is　more　suitable　for　elongated　structures　(jets)　identification.　Importantly,　the　method　presented　analyzes　the　evolutions　of　vortices　in　detail,　including　the　generation,　stretching,　and　merging　processes.　In　summary,　the　MKNN-based　clustering　algorithm　takes　particle　trajectories　as　input　data,　analyzes　the　evolution　of　relative　distances　between　particles　quantitatively,　and　carries　out　clustering　analysis　on　particles　according　to　trajectory　similarity　and　spatial　proximity.　The　combination　of　the　MKNN-based　clustering　algorithm　and　frame-invariant　identification　criteria　shows　great　potential　in　coherent　structure　identification　of　complicated　fluid　flows.