There's an interesting article on the decline of light crime novels by Colin Bateman over on the the Guardian books pages (you'd be forgiven for thinking that I don't read any other news website... that's cos I don't!)
It's a topic close to my heart, as I am working on a comedic SF crime novel, and Bateman ends with the heartening observation that "if you want to find something new and challenging, comic crime fiction is now the place to go." Perhaps I'll be able to catch a wave in twelve months or so, whenever I get it finished (about 12% through, at the mo, by my reckoning).
However, I don't think he quite answers the question in the standfirst, "when and why did it [crime fiction] lose its sense of humour?"
I blame critics.
Now, obviously, I am something of a critical bod myself (and although lately in abeyance I'm heading back into critic territory again shortly, God help me) and so I don't want to be mistaken for an anti-intellectual type. And I don't think it's just a matter of critics, individually, being humourless, but I think the act of criticism pushes us towards more serious works. I can't recall who said it now (frustatingly, as it's a point I'm forever bringing up) but some famous critic (yes, I know) said that the emergence of literary criticism had led to the development of the lit crit novel, that is novels written by writers raised on lit crit and written (unconsciously, perhaps) to feed critical debate rather than just be good books (I'll leave the question about the difference between those for another time) (I'll also try and lay off the parentheses for a bit).
I think that happened to a lot of genre fiction during the sixties and seventies. Left to itself, it was funny and scatological and wild and zany, and mixed up in the same way as the readership. When critics started paying more attention to pop culture, I think pop culture got all nervous and self conscious and got all serious.
Look at Philip K Dick novels: when he thought he was writing trash, he gave us Ubik, The Three Stigmata of Palmer Eldritch and Do Androids Dream of Electric Sheep? When he suddenly realised he was a great writer all along, we got Valis and The Transmigration of Timothy Archer, fine books to be sure, but without the zany edge that characterises Dick at his best. Suddenly, we get more gnosticism and questions of identity and less arguing with the pay-as-you-go front door over whether the quarter you have to pay to leave your apartment is more in the nature of a charge or gratuity.
Back when I started my MA, and I introduced myslef as a sci fi writer, my tutor nodded sagely and said "Ah, speculative fiction"! I said no. I said that I'm a sci fi author, with all the whacky baggage that entails. Hopefully I can hang on to that and ride it to fame and fortune. And then I'll buy the bloody Guardian and the centre spread every day will be my fat, smug smiling face.