Good old assert. Such a well meaning tool that can cause such a ruckus. Assert gets some bad press because it can be so jarring when it works. The documented behaviour for assert in C++ is:

In a typical Windows game, the abort part means the DUNH! system sound and an angry red dialogue box telling you the fun is over. If you're not a programmer, you might wonder why anyone would intentionally put this kind of thing in their code. If you are a programmer, you've probably noticed how similar an assert feels to an unhandled exception (i.e. a real crash). This causes everyone to lump asserts into the same mental category as "somebody broke the code".

To be fair, when an assertion fails it does means something bad has happened. Something that the code isn't built to handle. Something that if left unchecked will probably lead to a crash (though not always right away), maybe trashing some files in the process, wiping highscores from the server, or some other unpredictable peril. 

When something goes wrong in a program, it's like flying a helicopter as a part comes loose. Assert would be like freezing time right when that bolt falls off to figure out what went wrong. Not using assert is analogous to crashing and burning and then digging through the wreckage to figure out what happened. 

You can attach the debugger when your game asserts and take a look at what's going on - inspect variables, look at the call stack, and so on. Compared to tracing back from an often unrelated symptom, this kind of debugging is easy.

The assert macro in C++ does nothing in non-debug builds (i.e. when NDEBUG is defined). While this is good in principle, you can never know for certain that there isn't some obscure scenario that could trigger an assert, so it can be useful to have some around even in optimized builds. For this purpose, you probably want to write your own assert function/template/macro that ties into your error handling system and does something a little more graceful than abort(). Something like submitting a bug report would be ideal.

The catch: Assert is not a substitute for error handling. Asserts in production code should only be used to detect things that are impossible, or at least would be impossible if the code was written correctly. They're a tool for detecting coding errors.

Things that aren't coding errors are things like missing files, failed network connections, out of memory errors, to name a few. It's not the program's fault when these things happen. They are runtime errors that you have to detect and deal with yourself.

Assert is a useful tool for building solid, reliable code. The more you use it, the closer to the real problem you'll find yourself when something goes wrong. Don't be afraid to use assert - and if the DUNH! sound bothers you, you can always try changing your "Critical Stop" system to something less jarring.