Why won't video from ffmpeg show in QuickTime, iMovie or quick preview?

I've been using ffmpeg for the occasional video conversion for quite some time, and recently discovered that H.264 (libx264) videos generated by ffmpeg aren't working in iMovie or QuickTime (nor will they play or show a preview in Finder with the quick view feature).

If I generate videos using OS X applications which internally use ffmpeg (e.g. MP4tools or Adapter) the videos work fine, but not when I try to use ffmpeg directly, e.g. to convert a video:

# ffmpeg -i input.avi output.mp4

or to combine some images into a video:

# ffmpeg -f image2 -pattern_type sequence image-%04d.png output.mp4

iMovie just imports the video as black/blank, quick preview just shows the file info, (but no video preview), and Quicktime tells me it cannot play it:

What's going on?!