The supply of fresh minerals to Earth’s surface by erosion is thought to modulate global climate by removing atmospheric carbon dioxide (CO2) through silicate weathering. In turn, weathering of accessory carbonate and sulfide minerals is a geologically-relevant CO2 source, which may dampen or reverse the effect of silicate weathering on climate. Although these weathering pathways commonly operate side by side, we lack quantitative constraints on their co-evolution across erosion-rate gradients. Using stream-water chemistry across a 3 order-of-magnitude erosion-rate gradient in shales and sandstones of southern Taiwan, here, we demonstrate that silicate, sulfide, and carbonate weathering are linked: Increasing sulfide oxidation generates sulfuric acid and boosts carbonate solubility whereas silicate weathering kinetics remain constant or even decline, perhaps due to buffering of the pH by carbonates. On timescales shorter than marine sulfide compensation, CO2 emission rates from weathering in rapidly-eroding terrain are more than twice the CO2 sequestration rates in slow-eroding terrain. On longer timescales, CO2 emissions are compensated, but CO2 sequestration rates do not increase with erosion, in contrast to assumptions in carbon cycle models. We posit that these patterns are broadly applicable to many Cenozoic mountain ranges that expose dominantly siliciclastic metasediments.