President Francois Hollande and leaders from Germany, Italy, Israel, Turkey, Britain and the Palestinian territories among others, moved off from the central Place de la Republique ahead of a sea of French and other flags. Giant letters attached to a statue in the square spelt out the word Pourquoi?" (Why?) and small groups sang the "La Marseillaise" national anthem.
Some 2,200 police and soldiers patrolled Paris streets to protect marchers from would-be attackers, with police snipers on rooftops and plain-clothes detectives mingling with the crowd. City sewers were searched ahead of the vigil and underground train stations around the march route are due to be closed down.
The silent march - which may prove the largest seen in modern times through Paris - reflected shock over the worst militant Islamist assault on a European city in nine years. For France, it raised questions of free speech, religion and security, and beyond French frontiers it exposed the vulnerability of states to urban attacks.
Two of the gunmen had declared allegiance to al Qaeda in Yemen and a third to the militant Islamic State.
"Paris is today the capital of the world. Our entire country will rise up and show its best side," said Hollande in a statement.
Seventeen people, including journalists and police, were killed in three days of violence that began with a shooting attack on the weekly Charlie Hebdo known for its satirical attacks on Islam and other religions as well as politicians. It ended on Friday with a hostage-taking at a Jewish deli in which four hostages and the gunman were killed.
Overnight, an illuminated sign on the Arc de Triomphe read: "Paris est Charlie" ("Paris is Charlie").
Several London landmarks including Tower Bridge were due to be lit up in the red white and blue colours of the French national flag in a show of support for the event in Paris. Fifty-seven people were killed in an Islamist militant attack on London's transport system in 2005.
Hours before the march, a video emerged featuring a man resembling the gunman killed in the kosher deli. He pledged allegiance to the Islamic State insurgent group and urged French Muslims to follow his example.
"We're not going to let a little gang of hoodlums run our lives," said Fanny Appelbaum, 75, who said she lost two sisters and a brother in the Nazi concentration camp at Auschwitz. "Today, we are all one."
Zakaria Moumni, a 34-year-old Franco-Moroccan draped in the French flag, agreed: "I am here to show the terrorists they have not won - it is bringing people together of all religions."
Among many children brought along to the march, Loris Peres, 12, said: "For me this is paying respect to your loved ones, it's like family ... We did a lesson about this at school."
German Chancellor Angela Merkel, British Prime Minister David Cameron and Italy Prime Minister Matteo Renzi were among 44 foreign leaders marching with Hollande. U.N. Secretary-General Ban Ki-moon, Israel's Benjamin Netanyahu - who earlier encouraged French Jews to emigrate to Israel - and Palestinian President Mahmoud Abbas were also present.
Immediately to Hollande's left, walked Merkel and to his right Malian President Ibrahim Boubacar Keita. France has provided troops to help fight Islamist rebels there.
In a rare public display of emotion by two major-power leaders, cameras showed Hollande embracing Merkel, her eyes shut and forehead resting on his cheek, on the steps of the Elysee before they headed off to march.
After world leaders left the march, Hollande stayed to greet survivors of the Charlie Hebdo attack and their families.
While there has been widespread solidarity with the victims, there have been dissenting voices. French social media have carried comments from those uneasy with the "Je suis Charlie" slogan interpreted as freedom of expression at all cost. Others suggest there was hypocrisy in world leaders whose countries have repressive media laws attending the march.
The official estimate on attendance is due to be announced later. A 1995 protest against planned welfare cuts brought some 500,000-800,000 people onto the streets of the capital, while a 2002 rally against the far-right National Front's then leader Jean-Marie Le Pen afer he got into the run-off of that year's presidential election drew 400,000-600,000.
Twelve people were killed in Wednesday's initial attack on Charlie Hebdo, a journal know for satirising religions and politicians. The attackers, two French-born brothers of Algerian origin, singled out the weekly for its publication of cartoons depicting and ridiculing the Prophet Mohammad.
All three gunmen were killed in what local commentators have called "France's 9/11", a reference to the September 2001 attacks on U.S. targets by al Qaeda.
The head of France's 550,000-strong Jewish community, Roger Cukierman, the largest in Europe, said Hollande had promised that Jewish schools and synagogues would have extra protection, by the army if necessary, after the killings.
France's Agence Juive, which tracks Jewish emigration, estimates more than 5,000 Jews left France for Israel in 2014, up from 3,300 in 2013, itself a 73 percent increase on 2012.
Far-right National Front leader Marine Le Pen, whom analysts see receiving a boost in the polls due to the attacks, said her anti-immigrant party had been excluded from the Paris demonstration and would instead take part in regional marches.
In Germany, a rally against racism and xenophobia on Saturday drew tens of thousands of people in the eastern German city of Dresden, which has become the centre of anti-immigration protests organised by a new grassroots movement called PEGIDA.
A building of the newspaper Hamburger Morgenpost, which like many other publications has reprinted Charlie Hebdo cartoons, was the target of an arson attack and two suspects were arrested, police said on Sunday.
Turkish and French sources said a woman hunted by French police as a suspect in the attacks had left France several days before the killings and is believed to be in Syria.
French police had launched in an intensive search for Hayat Boumeddiene, the 26-year-old partner of one of the attackers, describing her as "armed and dangerous".