Tech Blog

Web Scraper Data Tool

By: Chris Dunn on Tue Mar 13, 2018 (3 min read)

A number of years ago I was migrating and merging a series of websites into a Content Management System.  We were attempting to organize a number of mixed technology sites, some hard-coded html, others ASP or PHP, under a single CMS.  Rather than manually translate each page into the new system, I decided to try parsing sites that followed a template pattern and import sections into the properties of the new system pages.  I developed a web scraping tool to extract that data based on a data map ...