Calling him a "father, husband, mentor and most of all a legend", West's family said the music lover passed away peacefully in his sleep on Thursday morning.
"Our family would appreciate respect and privacy during this difficult time," the family said in a statement released on Thursday night.
"Ken was big and noisy in life, but passed quietly and peacefully."
From comparatively humble beginnings headlined by recently world-famous Nirvana and the Violent Femmes in a sweaty Hordern Pavillion, the festival grew into an annual touring behemoth featuring many of the world's biggest artists.
West, along with business partner Viv Lees, created and guided the show that would grow to become a staple of the Australian and New Zealand summer from 1992 to its last edition in 2014.
The festival hosted a staggering list of international superstars such as Rage Against the Machine, Iggy and the Stooges, Beastie Boys, Red Hot Chili Peppers, Kanye West, Muse and Metallica, as well as Aussies heroes such as Powderfinger, Flume and Tame Impala.
Ken is survived by his wife, Cathy and son, Oliver.
In excerpts released earlier this year from an as-yet-unpublished book, West reflected on how what was originally meant to be a Violent Femmes tour blossomed into a huge day-long festival, at least partially off the back of Nirvana's explosion into the mainstream with the release of Nevermind.
Incredibly, the pair sold only 52 tickets on the first day, but demand eventually skyrocketed as the bill swelled to 21 acts across three stages.
"Stephen Pav was touring Nirvana at that time and as luck would have it, they were Violent Femmes fans," he wrote, in an excerpt published on his KenFest (at one point the name of the festival) website.
"Nevermind had been out since late September but was still only on the alternative charts. Soon after we went on sale on November 18 Nirvana exploded worldwide and eventually made the first BDO the hottest ticket in the country."
West, who believed the event would have sold out even without Nirvana's breakout success, said he even got a call from then prime minister Paul Keating trying to buy tickets for his kids.
British singer-songwriter Billy Bragg paid tribute to West on Thursday night.
"Ken made the whole thing like a party for punters and artists alike," he said, on Twitter.
"UK bands called it the Big Day Off.
"Lots of fun memories hanging with Ken and the BDO crew."
Veteran music writer Bernard Zuel described West as an "absolutely crucial part of the music business".
"(He was) the source of so many discoveries, an enthusiast, a wonderful nutcase," he said, on Twitter.
"He changed things."
Writing in his book excerpts, West described his role running the BDO as the "greatest job in the world".
"Working and partying with incredible artists who motivated me to create beautiful complex events," he said.
"Human alchemy was taken to its limits. Continually evolving and growing for over 20 years.
"Five million people attended those shows, that's a lot of joy."
