In order to explore the microbial communities and functions in distinct environments of acid mine drainage (AMD) ecosystem and fully comprehend the patterns of AMD formation and development, high-throughput sequencing technology was used to study the prokaryotic community composition in AMD puddles and surrounding rills in a mining area in Mengzi, Yunnan Province, China. By combining with physicochemical properties of the samples, we distinguished the key factors affecting the community structure and analyzed the environmental functions of the microflora. We discovered that the main phyla in AMD puddles were Euryarchaeota, Proteobacteria (including the class α-, γ- and δ-Proteobacteria), Nitrospirae, Firmicutes, Actinobacteria, and Acidobacteria. Community structure of the AMD puddles apparently differed from that of the surrounding rills. Microbial diversity was significantly positively correlated with pH, whereas the relative abundance of Thermoplasmata was negatively correlated with pH and this class might play a predominant role in community structure. There was high relative abundance of the genus Ferroplasma (6.60%–86.34%) in different samples of AMD puddles. Acidithiobacillus spp. were the major iron- and/or sulfur-oxidizing bacteria in AMD solutions and sediments, whereas relative abundance of the obligate iron-oxidizer Leptospirillum spp. was lower, and Ferrovum spp. were almost only present in the AMD solutions. In addition, acidophilic or acid-tolerant heterotrophic bacteria were widely distributed in the AMD solutions and sediments, which might promote the growth of iron- and/or sulfur-oxidizers and catalyze the oxidative/reductive dissolution of metal ores. Our results suggested that pH significantly impacted the microbial community structure of AMD environment by affecting the microbial diversity and microflora distribution.