This page does a very good study of how to manage mulitple currencies (actually, currencies, shares, etc.), with pro and cons of each method: https://www.mathstat.dal.ca/~selinger/accounting/tutorial.html
Note that you can adjust the different methods depending on your need, all of them are transposable to ledger (you might be interested in sections 5.11 to 5.20 in the ledger documentation too the method based on costs, if you’re discovering ledger: https://ledger-cli.org/doc/ledger3.html#Posting-cost )
On my side, I handle shares with costs method (so that I can track their prices easily), and "legal currencies" with the last method (trading buckets), since in my country those exchanges are not taxable; and very often I don’t want to care about conversion between legal currencies, I’m only interested in spending them
Note: the last method is very similar to your current situation, which seems perfectly sane