It would be foolish to underestimate the effort Bob Dylan has put into making himself rock music's greatest enigma. Rarely granting interviews, he also uses his influence to dissuade others from spilling any beans about him. "Bob really doesn't like people he knows to talk about his life," admitted Roger McGuinn, whose performances of Dylan's songs with The Byrds in the 60s were crucial in helping Dylan break through to a popular audience. When Dylan's current manager, Jeff Rosen, is approached for a few comments, he replies that not only will he not talk, he would prefer it if his name isn't even mentioned. He is just a "functionary", he explains, who "helps out with things". Neil Young's manager, Elliot Roberts, has known Dylan for decades, but he wouldn't comment either. "I know Bob pretty well," he agreed, "but I know that if I talk to you about him, I won't know him well any more."
Dylan once went some way to explaining himself to Jann Wenner of Rolling Stone magazine. "People don't understand that the press just use you to sell papers," he reflected. "They misquote you all the time. It hurts because you think you were just played for a fool. And the more hurts you get, the less you want to do it."
Yet Dylan's furtiveness seems to have gone beyond mere media-wariness and evolved into a Great Game worthy of a John le Carré espionage novel. Now that Dylan no longer commands the cultural high ground he dominated in his heyday, he will at least spin out his lingering mystique for all it's worth. Therefore the Bob Dylan currently touring Britain and Europe will not be going out of his way to explain his methods or his motivation, but he will doubtless feel reassured that the pendulum of public sympathy has recently swung back towards him.
At the end of the 80s, Dylan's career had reached its nadir. Having driven away his audience in droves with his bewildering born-again Christian period, he had moved on to discs like Knocked Out Loaded and Down In The Groove, which seemed pointlessly removed from either a contempo rary audience or fans of the classic Dylan of 20-odd years earlier. His appearance as a grizzled rock'n'roll veteran in the 1986 movie Hearts Of Fire looked like grotesque self-parody, almost as much as his membership of the Travelin' Wilburys, a wretched project featuring a bunch of senior citizens playing a Saga Holidays pastiche of country-rock. It seemed that the bones of Dylan's legacy had been left lying in the dust for rock historians to squabble over.
But what a difference a decade can make. Dylan rang in the new millennium with the song Things Have Changed, from the soundtrack of the Michael Douglas movie, Wonder Boys. The song's sly delivery and mordant surrealism stirred memories of a younger, hipper Dylan. In the accompanying video, dressed in a frock coat and straw boater and carrying a gold-topped cane, he put on a wry display of long-suffering bemusement, as if to say "I may be a genius, but look what I have to put up with." Then, last May 15, he was presented with the Polar Music Prize in Stockholm, an honour he shares with the likes of Stevie Wonder, Bruce Springsteen, Joni Mitchell and the classical violinist Isaac Stern.
If one single event could be identified as triggering the refurbishment of Dylan's fading reputation, it was his 1997 album, Time Out Of Mind. A haunted collection of blues, rockabilly and ballads apparently reaching out from beyond the grave, it confirmed that he hadn't, after all, lost the ability to hypnotise his listeners, nor to mine an indefinable sense of spiritual profundity from the simplest of musical tools. The record touched people like nothing he'd released in years.
Greil Marcus, that most professorial of rock critics who has made a continuing study of Dylan's work, taught a class at Berkeley called Prophecy in the American Voice, in which he invited his students to discuss Dylan's three most recent albums along with the evangelical speeches he had delivered from various concert stages during his born-again phase in the 1980s. "The students were absolutely enthralled by Time Out Of Mind," Marcus recalls. "They found it disturbing, mysterious, threatening, a little scary and utterly fascinating."
It seemed to Marcus's class that the album represented a delayed reaction to Dylan's evangelical period and whatever spiritual crisis had precipitated it. "They heard it as a record of abandonment," Marcus continues. "They said 'this is not about being abandoned by some woman. This record is about being abandoned by God, and it's really for the first time since this event coming to terms with rage and sorrow'. It was not something that I would ever have thought of, that that was how Time Out Of Mind works, but it seems a tremendously rich way of hearing it."
The fact that the disc could stimulate such a complex response might offer an answer to the question whether, 40 years after Dylan first travelled from Hibbing, Minnesota to New York, at the start of his long trail from folk singer to rock's greatest conundrum, his work can matter to listeners who weren't brought up listening to him either first hand or via parents or siblings. Not that Dylan seems to be holding his breath. "I can't say that it's never been my turn commercially, and that I've never known the feeling of having a record top the charts, because it wouldn't be true," he said, after the release of Time Out Of Mind. "But to have it again? I'm not really counting on it, 'cos I don't want to set myself up for a disappointment."
Greil Marcus cautions that "he doesn't necessarily mean anything to somebody younger, and there's no reason that he should if he's not able to seduce them now. That doesn't necessarily mean seducing them with new music. Somebody could bump into The Freewheelin' Bob Dylan [from 1963] and fall down the rabbit hole of his career that way."
It was this earliest incarnation of Dylan that Marcus first happened upon, when he went to see Joan Baez playing a folk concert in a field in New Jersey in 1963. "About halfway through she said I have a friend here I'd like to bring out, and this scruffy, dusty-looking character comes out with his shoulders kind of hunched. He sang a couple of songs with her and a couple of songs by himself, and then he left. That was the last time I was ever interested in Joan Baez. I'd never heard of him and I didn't quite get his name right, but I was at the record store the next day looking for something by him. There was a twisted labyrinth of possibility in that voice that I'd never encountered before, and I was just instantly captivated by him."
The Dylan that Marcus saw that day was a man ablaze with self-transforming ambition, in transit between the hobo ballads and dustbowl songs with which he first made an impression and the protest songwriter about to rewrite the rules of American folk music. Yet Dylan might have been a rock'n'roller from the very start, had he pursued the early enthusiasm for Little Richard-style piano thumping he displayed in Hibbing school bands like the Golden Chords and Elston Gunn & the Rock Boppers. In his 1959 high school yearbook, Dylan claimed his ambition was to "join the band of Little Richard".
He was born Robert Allen Zimmerman on May 24 1941 in Duluth, Minnesota. When he was six he moved with his parents to Hibbing, near the Canadian border. Robert and his brother David were the only children of Abe and Beatty Zimmerman, and their father was the proprietor of an electrical and furniture store. There was a primitivism about the wilds of northern Minnesota which stamped itself in Bob's imagination. "Every once in while a wagon would come through town with a gorilla in a cage or, I remember, a mummy under glass," Dylan reminisced. "It was a very itinerant place - no interstate highways yet, just country roads everywhere.There was an innocence about it all, and I don't recall anything bad ever happening."
But if the adult Dylan recalls Hibbing as a rural idyll, his teenage self was desperate to escape to the big city. He enrolled at the university of Minnesota in Minneapolis in 1959, though he focused on the blues and folk music in the local beatnik bars rather than on academic subjects. He changed his name to Dylan - believed by some to be a homage to the Welsh poet Dylan Thomas - and began to work up his own folk-singer act, but it was on a trip to Denver in 1960 that his ambitions began to crystallise. He began consciously to model himself on the protest singer Woody Guthrie, learning Guthrie's songs and even imitating his Oklahoma accent. Certain that he wanted to pursue a musical career, Dylan set off for New York where he arrived in January 1961.
His vagabond image and apparently "authentic" rural whine made an instant impact on the seething folk circuit in Greenwich Village. He seized every opportunity to embroider his personal mystique, exhibiting a rare gift for self-mythology as he spun out a skein of tall tales about his family and background. He claimed he had run away to a travelling carnival when he was 13, denied he'd ever been called Zimmerman, and professed to have studied with an assortment of obscure blues and folk singers. He presented his arrival in New York as partly a pilgrimage to visit the dying and bedridden Woody Guthrie.
Dylan's self-penned new identity was good enough to fool the experts. The folk singer Ramblin' Jack Elliot, another composer from whom he had borrowed liberally, commented that "there was not another sonofabitch in the country that could sing until Bob Dylan came along. Everybody else was singing like a damned faggot." In a pivotal New York Times review, Robert Shelton waxed euphoric about how Dylan evoked "the rude beauty of a Southern field hand musing in melody on his porch". The review was spotted by talent-scout John Hammond, who signed him to Columbia Records in autumn 1961.
With hindsight, Dylan's protest-singer period seems no more than a blip in his long career, but between 1962 and 1964 he became the patron saint of the Civil Rights movement. Songs like Blowin' In The Wind and The Times They Are A-Changin' mirrored the passion for social justice that was sweeping contemporary American youth, and are still among his best known. Yet it seems that Bob already foresaw an electric future, since as early as 1962 he recorded Mixed Up Confusion with a rock'n'roll backing. Some suspect that it was his brilliantly machiavellian manager, Albert Grossman, who grasped that Dylan's immediate future lay in the folk-protest sphere, and Confusion was hastily deleted.
Dylan's public conversion to amplifiers and electricity has become an essential bullet-point in any potted history of rock. It is frequently claimed that his performance with an electric band at the 1965 Newport Folk Festival was booed and jeered by the folk diehards, but Al Kooper, Dylan's organist on that occasion, says it wasn't like that at all. Like A Rolling Stone was already a radio hit, and the crowd booed because they wanted more of the rocking Dylan, not less. "Damn right, they booed," Kooper recalled. "But not at Bob - rather, at whoever was seemingly responsible for yanking him offstage after 15 minutes." Kooper, who bluffed his way into playing the Hammond organ during recording sessions for Highway 61 Revisited and almost inadvertently helped create what Dylan would call his "wild mercury" sound, described Bob's pivotal influence at his mid-60s peak. "In 1965, sessions for Dylan albumsmight as well have been strategy meetings for the direction the new alternative culture would take."
For all Dylan's brilliance at exploiting the way musical and social trends were shifting, he wouldn't have been able to make his crossover into rock without listeners eager to go with him. In the same way that he had been thrilled by such musical experiments as The Animals' rock version of House Of The Rising Sun and The Beatles' continuing advances in songwriting and use of studio technology, a wider audience was urging the musicians to take more risks. "It was obvious to me that The Beatles had staying power," Dylan commented. "I knew they were pointing the direction of where music had to go."
He benefited hugely from Albert Grossman's support. "The only person who was as indecipherable as Grossman was Dylan - and their closeness and similarities led to a great deal of speculation within both the music business and the folk community about who had invented whom," wrote Fred Goodman in his book The Mansion On The Hill. "The reality was that they were unusually well-matched soul mates."
Grossman parlayed Dylan's creative genius into awesome commercial clout. Grossman was the first rock manager to realise how desperately record companies needed creative artists in that era of dawning youth consciousness, and he used that leverage to cut deals giving his acts unprecedented earnings and artistic control. Meanwhile, he was shrewd enough to see the benefit of having Peter Paul & Mary, also from his management stable, sing a sugary, chartbusting version of Blowin' In The Wind to boost Dylan's commercial profile.
Dylan and Grossman severed their professional relationship in 1970, but the superman of folk-rock has regularly relied on third-party assistance as a catalyst for his strongest work. At different times, The Band and Tom Petty's Heartbreakers gave him a scope and discipline which allowed his music to thrive, while producer Daniel Lanois has been at the helm of his best recent albums, Oh Mercy and Time Out Of Mind.
"He gets people to kill themselves for him, and that's a mark of a certain kind of talent," comments Donn Pennebaker, who saw the Dylan operation from the inside when he made his 1965 tour documentary, Don't Look Back. "But I think it frustrates Dylan sometimes. He gets people around him, like The Band, and then he has to run away from it because it's pinning him down. He has to keep being Dylan, which he doesn't want to do. He wants to be something more ephemeral."
Dylan's power to play with image and identity can still mesmerise other performers, even if he no longer inspires the massed legions of impersonators that erupted in his wake during the 60s and 70s, when everybody from David Bowie to Bruce Springsteen and Patti Smith was hailed as a New Dylan. One contemporary artist captivated by the myth and magic of Bob is Tim Burgess of The Charlatans, whose own songs feature the occasional sly quote from the bard of Minnesota. As Burgess sees it, Dylan's process of perpetual reinvention has never stopped, whether he happens to be in or out of fashion. "I love Time Out Of Mind, because for the third or fourth time in his career he's found a new voice. I think maybe a few LPs before that one he was discovering something but hadn't quite got there. Then with Time Out Of Mind he's right on the button with his new voice and his new image. I think he did it with Nashville Skyline in the late 60s, then again with Blood On The Tracks, and again with Oh Mercy. It's like decades apart, but he's got a new voice and I think that's incredible."
Critics habitually forget that it's a fatal mistake to identify Dylan with the characters and circumstances portrayed in his songs, and that the purpose of art isn't merely to clone experience, but to transform it. With 1975's Blood On The Tracks, the anguish of the breakup of his marriage to the former model Sara Lowndes pushed Dylan to fashion a set of songs which amounted to a philosophical re-imagining of his whole life, portraying it as a kind of mythic quest signposted with symbols and riddled with ironies. His poetic distillation of love, memory, coincidence and fate was as close as rock music has come to great literature.
"I don't listen or read or watch movies looking for any kind of autobiographical subtext," Greil Marcus points out. "Art to me is about something that is imagined. It's a world that does not really exist that you're allowed to enter. To me, whether or not Dylan is creating a persona, I'm hearing one."
Marcus has been particularly absorbed by the music Dylan made during 1967 at a hideaway in upstate New York. Following his conquest of rock'n'roll Everest with the brilliant trio of albums which culminated with Blonde On Blonde, and a drug-fuelled schedule of international touring, Dylan had retreated into the countryside near Woodstock with his wife Sara. A motorcycle accident in July 1966 left him with concussion and some cracked vertebrae, and during his recuperation, Dylan realised how urgently he needed to change his both-ends-burning lifestyle. "I thought I was just gonna get up and go back to doing what I was doing before," he said later, "but I couldn't do it any more."
He had married Sara Shirley Lowndes in New York in November 1965, apparently captivated by her aura of spiritual calm and aloofness from the nightclubbing crowd he hung out with. The Dylans hurled themselves into domesticity and parenthood. Jesse was born shortly before the motorcycle crash, joining Maria, Sara's daughter by her previous marriage. By 1969, Anna, Sam and Jakob had swelled the Dylan clan. It would seem to Dylan's bewildered fans that private contentment had fatally diminished Bob's artistic drive, since this was the period when he would produce Nashville Skyline and Self Portrait, albums which seemed almost parodies of the settled, rural life the Dylans were leading. According to journalist Al Aronowitz: "Bob and Sara grew to be one of the tightest twosomes I've ever known. In the years following his motorcycle accident, Bob acted like a romantic cornball when he was with her."
However, even as Dylan grew a beard and threatened to turn into a beatnik Bing Crosby, he still found time during 1967 to make the Basement Tapes, a sprawling array of songs recorded in a rented house with The Band. They caught him exploring a range of folk, blues and country styles. Turning away from the technical and stylistic advances which were expected from rock music at the time (typified by the Beach Boys' Pet Sounds or The Beatles' Sgt Pepper), he had immersed himself in a timeless well of traditional American songwriting. "He would pull these songs out of nowhere," said Robbie Robertson, The Band's guitarist. "We didn't know if he wrote them or if he remembered them. When he sang them, you couldn't tell."
In its way, Dylan's retreat into traditional musical forms was as much a signpost of the times as his earlier groundbreaking accomplishments. With Dylan's sanction, rock bands like the Flying Burrito Brothers, Crosby Stills & Nash and The Eagles suddenly felt they had permission to play airy, melodic country rock. Throughout the 70s, as the hangover from the 60s set in, vocal harmonies and pedal steel guitars became strangely reassuring after years of screaming psychedelic guitars and acid-rock rambling.
With hindsight, it's remarkable how Dylan's influence pervaded even projects in which he ostensibly had no involvement, as if nobody dared make a move without Bob's tacit approval. When Peter Fonda and Dennis Hopper were making their breakthrough counterculture movie Easy Rider in 1969, they wanted Dylan to contribute to the soundtrack. But it was Roger McGuinn who wrote and sang The Ballad Of Easy Rider for the film.
"Peter Fonda wanted a song written for the movie, so he took it to New York and screened it for Bob," McGuinn says." Dylan wrote some notes on a cocktail napkin, handed it to Peter and said 'Give this to McGuinn, he'll know what to do with it'. I got the napkin, wrote the melody and finished up the lyrics. When the song came out, I gave Bob a 50% writer's credit, but he called me and asked me to take it off. He said he didn't need the money."
Over subsequent decades, Dylan's power to influence mass popular movements has inevitably waned, even though his throwaway remark from the stage at Live Aid led directly to the formation of Farm Aid to benefit struggling American farmers. Depressingly, he has often resembled a derelict figure stumbling through mediocre concerts and churning out records in which he palpably has no interest, while what we can glean of his private life confirms the notion of a rootless loner doomed never to recapture the sense of belonging that his marriage to Sara once afforded him.
Since his divorce in 1977, a string of lovers has drifted in and out of his various homes in Malibu, Minnesota and New York, among the more conspicuous being Columbia Records executive Ellen Bernstein, gospel singer Clydie King, actresses Mary Alice Artes and Sally Kirkland, estate agent Britta Lee Shain and a French painter named Claude-Angele Boni. In 1994, Ruth Tyrangiel served Dylan with a $5m "palimony" suit, claiming that they had lived virtually as husband and wife for 17 years, but Dylan settled privately for a much-reduced sum.
In 1997, his paranoia about publicity must have soared off the dial when news leaked out of a proposed kiss-and-tell memoir - so far unpublished - by yet another ex-girlfriend, Susan Ross. This would depict him, allegedly, as a creatively bankrupt drinker, and even contain details of "secret marriages" and consequent offspring. It seems that Dylan may have married his erstwhile backing singer Carolyn Dennis in 1980 and fathered two children with her, but he has managed to sustain a total news blackout on the issue.
Yet whatever personal chaos may teem behind the Dylan legend, nothing can tarnish the grandeur of his reputation as the great folk-rock-poet from the 60s. The world has fixed for all time what it wants him to stand for. Wherever he goes, regardless of how dismal his previous couple of albums may have been, civic dignitaries or august panels of his peers are eager to heap fresh honours on him. In 1982, he was inducted into the Songwriters Hall of Fame ("this is pretty amazing because I can't read or write a note of music," he confessed). In 1989, Springsteen inducted him into the Rock And Roll Hall Of Fame, asserting that Dylan "had the vision and the talent to make a pop song that contained the whole world".
In 1990 Jack Lang, the French culture minister, made him a Commandeur de l'Ordre des Arts et des Lettres in Paris. In 1991, he was awarded a Lifetime Achievement Award at the 33rd Grammy Awards. In 1992, a parade of illustrious rock'n'roll dignitaries turned out for the Bob Dylan 30th anniversary celebration concert at New York's Madison Square Garden. And in 1997, Dylan knocked loudly on heaven's door by playing to Pope John Paul II in Bologna, as millions watched on a Euro-wide telecast.
HE once presided over the birth of a new artistic medium. Now, as it slides into history, Dylan is being enshrined along with it. It may be that we're coming to the end of the whole tradition of art and artists that he has represented, even if his son Jakob is making a belated stab at rock stardom with his own band, The Wallflowers.
"I think we're coming to the end of the Byronic age, I'm convinced of this," suggests Pennebaker. "The role of the Byronic hero is no longer valid - marching through life with his noble brow, creating this forceful art. In the 60s, kids in high school wanted to be in rock'n'roll bands, but now they want to go into internet start-ups. They want to get hold of some aspect of power that's suddenly available to everybody, when for a long time only artists seemed to have a grab at it.
"But Dylan does have a funny kind of spiritual hold over a whole generation of people, and he's been able to keep it even though they hardly ever play his goddamn records on the radio. It's some kind of myth from the past, but it holds people." For anybody who originally thrilled to the boundary-breaking magnificence of Dylan's finest work, this notion of him as the ageing object of train-spotter's desire is difficult to adjust to. The sarcastic, razor-edged Dylan of 1966 would have gleefully put to the sword the kind of trivia-obsessed completists who now follow him around the world. "Forget the dead you've left, they will not follow you," he once wrote. He may be having second thoughts about that now.
• Bob Dylan UK tour dates - today Cardiff International Arena; tomorrow and Monday Portsmouth Guildhall; October 5 and 6 Wembley Arena.
Life at a glance: Robert Zimmerman
Born: May 24 1941, Duluth, Minnesota.
Educated: Schools in Hibbing Minnesota; University of Minnesota, Minneapolis.
Married: November 22 1965 Sara Lowndes (four children - Jesse, Anna, Sam, Jakob - and Dylan adopted Sarah's daughter Maria from her previous marriage), divorced 1977.
Albums include: Bob Dylan (1962); The Freewheelin' Bob Dylan (1963); The Times They Are A-Changin' (1964); Bringing It All Back Home, Highway 61 Revisited (1965); Blonde On Blonde (1966); John Wesley Harding (1967); Blood On The Tracks ), The Basement Tapes (1975);
Slow Train Coming (1979); Knocked Out Loaded (1986); Oh Mercy (1989); Time Out Of Mind (1997); Live 1966 (1998).
Films: Don't Look Back (1965); Pat Garrett And Billy The Kid (1973); Renaldo And Clara (1978); Hearts Of Fire (1986).
Some publications: Tarantula (1971); The Songs Of Bob Dylan 1966-75; Writings And Drawings (1973); Saved (gospel speeches, 1990); Drawn Blank (1994).
Awards: Hon DMus, Princeton (1970); Songwriters Hall of Fame (1982); Rock and Roll Hall of Fame (1988); Commandeur de l'Ordre des Arts et des Lettres (1988).