Vermaelen started his career in his home country at Germinal Ekeren, which later changed its name to Germinal Beerschot after a merge.
In 2000 he joined the academy of Dutch side Ajax. He made his professional debut on February 15, 2004 in a 2-0 away win over FC Volendam. It was however the only match he played during the season in which Ajax won the Eredivisie championship. He was sent on loan to RKC Waalwijk for the 2004–05 season. At RKC he was not a first team regular, but had 13 appearances in which he scored two goals.
His breakthrough came with his return to Ajax and won the KNVB Cup. This performance led him into the Belgium national football team. The Johan Cruijff Schaal and another KNVB Cup were added in the following season and in 2007 he won the Johan Cruijff Schaal for the second consecutive time. Vermaelen served as captain of Ajax for the 2008-09 season.
Vermaelen joined Arsenal from Ajax for an initial fee of €10 million (£8.45m), rising to €12 million (£10.1m) on 19 June 2009.