Mediterranean fluvial ecosystems are subjected to ever-increasing water demands and to a wide variety of other human impacts with potential negative effects on riparian vegetation. Nevertheless, few studies have compared the importance of human versus natural factors in structuring riparian vegetation in this region, particularly in semi-arid areas. We examined basin-scale responses of riparian vegetation to the main environmental gradient extracted by canonical correspondence analysis in two semi-arid Mediterranean rivers in southern Spain, considering different community attributes (woody, herbaceous, functional groups, and exotics species) and using different metrics (species composition, richness, and cover). The results show two main environmental gradients for riparian vegetation: a prime altitudinal gradient of increasing salinity towards the lower sites, and a second gradient of increasing water pollution and human physical impact parallel to the hydroperiod shortening. Species richness of woody and herbaceous life forms, and cover of woody species, reacted negatively to the increasing salinity downstream, a pattern that was driven by most functional groups. This is consistent with the key role of salinity for vegetation. Furthermore, as for other semi-arid riparian ecosystems, the decline in rainfall coupled with higher temperatures and evaporative stress at the lower elevation open-canopy sites may have constrained herbaceous species. Woody richness and cover declined with shorter hydroperiod and increasing human impacts, a response that was driven by mesic and hydric species. These results agree with the idea that surface flow permanence is an outstanding limiting factor for woody vegetation in semi-arid rivers, possibly exerting more influence on species richness than flood disturbance. However, herbs showed very little response to drought or human impacts compared to woody species, which may be related to the higher recovery rate from disturbance of herbaceous life forms, due to their shorter life-span and higher colonization rates. Generally, exotic species were positively affected by dissolved nutrients (N and P), especially exotic herbaceous richness and cover by nitrogen. As in other Mediterranean rivers, percentage richness of exotics in our rivers was substantially lower compared to fluvial ecosystems in other regions.