Speech Inventories

"Such an association suggests that inventory size and structure may be related in other ways as well. A simple form of such a hypothesis would propose that segment inventories are structured so that the smallest inventories contain the most frequent segments, and as the size of the inventory increases, segments are added in descending order of their overall frequency of occurrence. If this were so, all segments could be arranged in a single hierarchy. Such an extreme formulation is not correct, since no single segment is found in all languages. But if we add a corollary, that larger inventories tend to exclude some of the most common segments, then there is an interesting set of predictions to investigate. We may formulate these more cautiously in the following way: a smaller inventory has a greater probability of including a given common segment than a larger one, and a larger inventory has a greater probability of including an unusual segment type than a smaller one."
(Patterns of Sounds, Ian Maddieson)

I would say that there is a convergence towards a inventory size with a number of segments between 20 and 40 and using a restricted set o segments that tends to be common among languages those languages. As the language gets further away from this zone, we cannot say much about what would be its inventory.