CBSbot

CBSbot is the name of a computer program used by Statistics Netherlands to gather data from the internet for statistical purposes. When CBS has visited your website, the name “CBSbot” will show up in the log files of your website.

Operation and identification

Statistics Netherlands uses the CBSbot to gather data for statistical production or in studies exploring new options for statistics. If CBSbot is used for statistical production, and substantial amounts of data are collected periodically, then CBS will also inform you in writing. If smaller amounts of data or exploratory studies are involved, then CBS will inspect if your site has any robots.txt files so that we know which parts we can and cannot visit (see the Robots Exclusion Protocol). To make it possible for you to distinguish a visit from CBSbot from a visit by normal users, CBS identifies it CBS by means of the term “CBSbot” as part of the User-Agent string.

Operation

CBS wishes to limit the burden to your site to an absolute minimum. Therefore CBSbot operates preferably at quiet times, mainly at night. Furthermore CBSbot will only collect those parts of the pages it requires for statistics, usually no images or other graphic elements. Finally, subsequent requests will be sent with some pauses in between.

Privacy

CBSbot only visits publically accessible parts of the internet. CBS deals with the data gathered with the utmost care, according to the Personal Data Protection regulations that apply to CBS.

European guidelines:

Statistics Netherlands thus complies with the European guidelines for collecting data from the web for statistical purposes: European Web Content Retrieval Guidelines

Any questions? Please let us know

If you have any questions, or a request to have CBSbot visit your site in a specific way (for instance the time of the visits or the times between requests), or another kind of data exchange, please contact us through the contact form at the CBS website (subject CBSbot).