Ok now this is an interesting comparison you bring up, let's see:

Nitrous Oxide:

N~2~O, that's it, a simple molecule. It's used constantly in medical contexts, where its safety profile is so well understood and its action is so consistent, it's given to pregnant women in childbirth to manage pain and anxiety. Binds to one receptor, clear mechanism of action. Possibly the most well-understood mind-altering substance in existence.

Cannabis:

a plant - a complex multicellular organism. Consumers inhale/digest/absorb an entourage of dozens of different compounds bundled together. Countless strains, cultivars and growing methods, with no two being the same, all with their own unique genetics and chemical "fingerprint". Preparation and consumption method adds even more complexity. Multiply that by 100s of millions of consumers worldwide, each one with their own human body, no two the same. An unfathomable level of complexity, which science is actively researching on a continual basis. Binds to many different receptors/systems when consumed.

We continue ironing out the details of long term safety and how best to harness it medically. And, guess what, responsible users are more than willing to change their mind and their habits when new information is uncovered. And yet, it is still mostly safe for most people, and still safer than most other recreational substances by a wide margin. There is no practical LD50 for cannabis or its constituents. The "lethal dose" would be so comically large, it would be physically impossible to consume in time to do any harm at all. You're orders of magnitude more likely to be crushed to death by a giant weed bale accidentally falling over on top of you than you would be consuming it.