Background: Dementia is a major public health problem, and repeated cognitive data from large epidemiological studies could help to develop efficient measures of early prevention. Data collection by self-administered online tools could drastically reduce the logistical and financial burden of such large-scale investigations. In this context, it is important to obtain data concerning the comparability of such new online tools with traditional, supervised modes of cognitive assessment. Objective: Our objective was to compare self-administration of the Web-based NutriNet-Sante cognitive test battery (NutriCog) with administration by a neuropsychologist. Methods: The test battery included four tests, measuring, among others aspects, psychomotor speed, attention, executive function, episodic memory, working memory, and associative memory. Both versions of the cognitive battery were completed by 189 volunteers (either self-administered version first, n=99, or supervised version first, n=90). Subjects also completed a satisfaction questionnaire. Concordance was assessed by Spearman correlation. Results: Agreement between both versions varied according to the investigated cognitive task and outcome variable. Spearman correlations ranged between .42 and .73. Moreover, a majority of participants responded that they "absolutely" or "rather" agreed that the duration of the self-administered battery was acceptable (184/185, 99.5%), that the tasks were amusing (162/185, 87.6%), that the instructions were sufficiently detailed (168/185; 90.8%) and understandable (164/185, 88.7%), and that they had overall enjoyed the test battery (182/185, 98.4%). Conclusions: The self-administered version of the Web-based NutriCog cognitive test battery provided similar information as the supervised version. Thus, integrating repeated cognitive evaluations into large cohorts via the implementation of self-administered online versions of traditional test batteries appears to be feasible.