The worst bug I have fixed had no stack trace.
On easy.jobs, submitting a job application kicks off a tail of work: in-app notification, confirmation email, pipeline assignment, AI scoring, a Slack alert, a Zapier webhook. For a period in production, that whole tail stopped. Applications were saved. Candidates saw the success screen. Employers heard nothing. Nothing was logged, because from the code's point of view nothing had gone wrong.
That is the shape I now watch for: the system is not failing, it is succeeding at the wrong thing.
The root cause was mundane, as they usually are. The lesson was not, and it changed how I build anything with side effects.
Side effects need their own assertions. A test that says "the application was saved" is not a test that says "the employer was told". I now write the second kind, and in production I record the absence of an expected effect as an event in its own right. "Zero notifications sent for an application" is a metric that alerts.
Silence is a bug class. Catch-all exception handlers, queued jobs with no failure hook, try/catch blocks that log at debug: each is a place where a failure can become a success. I audit for them the way I audit for SQL injection.
Let the queue keep the message. The deepest version of this problem is a failed job that gets deleted. When I later moved work onto AWS SQS, I did not want to rely on remembering to configure retries in every job class. SQS already has redrive policies and dead-letter queues; Laravel just was not using them. So I wrote laravel-sqs-redrive, which detects the redrive policy and lets SQS route the failed job instead of deleting the message. The failure is preserved, visible and replayable.
None of this is clever. It is the difference between a system that tells you when it is broken and one that makes you find out from a customer.