BBC News 16 June, 2012Mr Barry becomes the third recipient of the award, which forms part of the Borders Book Festival in Melrose.
Judges said the book displayed "wonderful writing, which, as Walter Scott did in his time, shifts perception on a period in history".
Mr Barry said: "I'm uncharacteristically speechless."
The 56-year-old playwright, novelist and poet has been twice shortlisted for the Booker Prize for his novels A Long Long Way and The Secret Scripture, and his latest was longlisted last year.
On Canaan's Side traces the memories of an Irishwoman as she flees her homeland's troubles after World War One for a new life in the US.
It was up against The Sisters Brothers by Patrick de Witt, Half Blood Blues by Esi Edugyan, The Stranger's Child by Alan Hollinghurst, Pure by Andrew Miller and The Quality of Mercy by Barry Unsworth.
Extracts from the six shortlisted books were read by actor John Sessions at the award ceremony, where tributes were paid to Unsworth, who died on 5 June.
"I really was not expecting to win - just look at the other authors on the shortlist," said Mr Barry, who won the 2008 Costa Book of the Year for The Secret Scripture.
"My first encounter with Walter Scott was unlocking a trunk, in my grandfather's attic, which contained the 'Waverley' novels.
"I felt as if I was excavating a tomb. I think that is an appropriate way to encounter a writer - as if you were literally retrieving him from the damp and history of your grandfather's life."
The judges, who included television presenter Kirsty Wark, had previously said that they had "the strongest and closest shortlist for the prize so far" and admitted that it had been a tough choice.
"There was little more than a whisker between On Canaan's Side and the other five shortlisted novels, but it was its drive, and its sustained power, that persuaded us to award the Walter Scott Prize to Sebastian Barry," they said in a statement.
"A work of immense power, the book is muscular and complete, and the author wears his learning lightly. Every character is fully drawn and utterly memorable."
The Walter Scott Prize, sponsored by the Duke and Duchess of Buccleuch, distant kinsmen of Sir Walter Scott, is the fifth richest UK literary prize and, to qualify, novels must be set 60 years ago or more.