When David Bowie chose to break a decade's silence by releasing a single, Where Are We Now?, on his 66th birthday, dissenting voices were hard to find amid the clamour made by people eager to welcome him back. Some argued that the clamour was part of the problem: it drowned out the music, which perhaps wasn't worthy of the noisy excitement it had caused. The reason people were so thrilled Bowie was back, they suggested, was founded in the music he made in the 1970s, a decade when almost every new album he released was an astonishingly sure-footed leap forward into uncharted territory. But Where Are We Now? was no Heroes or Sound and Vision. Rather, it was a charming, fragile ballad. Indeed, it was not unlike the stuff he had been knocking out immediately before his retirement, when – presumably burned by the mixed response to his spirited 1990s attempts to seize the zeitgeist or dabble in the avant garde in the way he'd done so effortlessly two decades previously – Bowie settled on a more straightforward and comfortable kind of classicism. Had Where Are We Now? been released in 2004, they suggested, its reception would have been far more muted. People were welcoming back an exhilarating, distant memory of Bowie, rather than the reality.
They had a point, but Where Are We Now? still carried with it some intriguing possibilities for the subsequent album. The song found Bowie drifting nostalgically around his former adopted hometown, Berlin, suggesting that The Next Day might be that rarest of things, an autobiographical work from an artist whose nakedly autobiographical songs can be numbered on the fingers of one hand. Such speculation was bolstered by the self-referential artwork of both the single and album: the former offering an inverted photo of a skeletal Bowie onstage in 1974, the latter featuring the cover of 1977's Heroes with the title redacted and a white square covering the famous image of the singer in a pose modelled on a painting by Erich Heckel. Perhaps, came the excited suggestion, Bowie had spent the past decade ruminating on his past and was now issuing an album in lieu of a memoir.
Perhaps not. There are certainly a smattering of knowing sonic references to Bowie's past works. If You Can See Me features the eerie sped-up vocal effect found on 1970's After All and the climax of Hunky Dory's The Bewlay Brothers, while You Feel So Lonely You Could Die ends with a reprise of the drum beat that introduced Five Years, the opening track of The Rise and Fall of Ziggy Stardust and the Spiders from Mars. And there are moments when you wonder whether some of the characters in The Next Day's songs aren't founded in personal experience: certainly, the protagonist of Love Is Lost – appointed "the voice of youth", marooned abroad, cosseted by luxury but fearful and paranoid, "thinking like mad" – feels remarkably like a pen portrait of the poor, cocaine-ravaged soul in that inverted photo. But elsewhere The Next Day offers what you might call an index of Bowiean obsessions.
Reading this on mobile? Click here to view video
In the first 10 minutes alone, you get the terror of life in a dystopian dictatorship teetering on the brink of apocalypse; a feral gang of vaguely homoerotic juvenile delinquents smashing things up; the numbing isolation of stardom and the suggestion that stars themselves may actually be some kind of alien lifeforms "soaking up our primitive world". The mutual respect between Bowie and Scott Walker is well-documented – an effusive 50th birthday tribute from the elusive former Scott Engel famously reduced Bowie to tears live on Radio 1 – and it's Walker's latterday work that much of The Next Day resembles, at least in that the lyrics are so dense and allusive you occasionally feel in need of a set of York Notes to get through them.
The present writer spent an alternately illuminating and rather trying few hours attempting to unpick the lyrics of I'd Rather Be High. Perhaps the mention of "Clare and Lady Manners" drinking and gossiping about politics during wartime was a reference to The Coterie, the 1910s' equivalent of the subsequent decade's bright young things – its male membership ultimately decimated in the trenches – in which Lady Diana Manners played a leading role. Or perhaps it was a reference to Officers and Gentlemen, the second novel in Evelyn Waugh's Sword of Honour trilogy, in which a character based on Lady Diana Cooper (nee Manners) attempts to pull strings for Ivor Clare, a character facing desertion charges. That seems more likely – amid the subsequent lyrical references to the futility of conflict, there's a mention of Egypt, where Officers and Gentlemen is partially set – in which case, the song's overall message might be summarised as: Waugh – huh! – what is it good for? But what does Officers and Gentlemen – or, for that matter, The Coterie – have to do with the opening line's reference to Vladimir Nabokov's life in 1920s and 30s Berlin? Pausing only to wonder whether there's a certain cultural richness here that you just don't find in, say, the oeuvre of the Vaccines, or whether Bowie has earned himself the exalted position where one takes for cultural richness the kind of thing you'd ordinarily dismiss as agonising pretention – and to note that either is deeply impressive – the present writer gave up and decided to just enjoy the music.
This, it has to be said, is a relative doddle. Producer Tony Visconti has suggested that The Next Day is of a piece with 1979's Lodger and, as on that record, Bowie spends a lot of The Next Day experimenting with his vocal delivery, offering, among other things, a peculiar nasal drone on the title track and a doomy, tortured lowing that recalls Walker – him again – on the closing Heat. The dense web of screaming feedback that ends Where Does the Grass Grow? recalls the climax of Boys Keep Swinging, while the fantastic If You Can See Me has some of the relentless propulsion of Move On, although it's perhaps worth noting that the latter track was about the joyous freedom of travel, while the characters here are unable to escape the shadowy forces controlling their lives however far they run: "if you can see me, I can see you," reiterates the chorus, Big Brother-ishly. That aside, the comparison with Lodger might be pitching it a bit high. It's perhaps the least well-regarded album of original material Bowie released in the 70s, but that tells you more about the astonishing quality of the records that preceded it than it does Lodger itself. If there are some intriguing musical decisions on The Next Day – the honking baritone sax that gives the feral gang of Dirty Boys a curiously lurching, ungainly gait; the vocal rendering of the Shadows' Apache that constitutes Where Does the Grass Grow?'s hook, the ominous, fretless-bass-decorated shudder of Heat – there's none of Lodger's unfettered experimentalism, nothing as authentically bizarre as Yassassin or African Night Flight.
What The Next Day has that perhaps Lodger didn't is something more prosaic. Whatever else he's been doing, clearly at least some of the last decade has been spent carefully crafting inarguable tunes. Its melody shifting from weary sigh to frantic angst, I'd Rather Be High is utterly beautiful; The Stars (Are Out Tonight) supports its Brad-Pitt-is-an-alien thesis with a fantastic chorus, all the more potent for the fact that it takes an age to arrive; Valentine's Day is so deceptively sweet that the bleakness of its subject matter – another tyrant, bent on crushing the world beneath his heels – doesn't initially register.
Despite the lyrical density, The Next Day's success rests on simple pleasures, not a phrase you'd ever use to describe Lodger or Station to Station. You could argue that means the naysayers still have a point. For all the pointers it offers in that direction, The Next Day isn't the equal of Bowie's 70s work: but then, the man himself might reasonably argue, what is? Perhaps it's destined to be remembered more for the unexpected manner in which it was announced than its contents. That doesn't seem a fair fate for an album that's thought-provoking, strange and filled with great songs. Listening to The Next Day makes you hope it's not a one-off, that his return continues apace: no mean feat, given that listening to a new album by most of his peers makes you wish they'd stick to playing the greatest hits.
• The Next Day is released on 11 March