The constitution doesn’t say only the Supreme Court can decide something is illegal. The supremacy clause does give the SC the power to decide if a law violates the constitution, and if so, the constitution takes precedence after review.

The law, while it is enacted, defines what is legal as a tautology. If judicial review overturns a law, then such actions are obviously not illegal anymore because the law would not be in effect. But you aren’t going to avoid prison by arguing that the statutory basis of your charges has never been reviewed and is therefore in a quantum superposition of legality.

Which brings me to my earlier point: the constitution empowers the Supreme Court to judicial review for settling matters of constitutionality. Nowhere does it say that only the Supreme Court can make legal determinations. That’s more or less how Roe was overturned recently: the SC argued that an abortion ban is not unconstitutional, therefore the SC cannot nullify abortion ban laws. This means that lower courts can determine abortion to be illegal, according to their local laws.

The modern idea of the Supreme Court’s authority, which I referred to as a meme, comes more from Marbury v Madison than from the constitution.