CODEWITHSUNDEEP
information-extraction
DMor
DMor

Reputation: 789

How to connect a program to a (school) website to extract data

This is just brought up by a program created for Universities that pulls entire lists of available classes, codes, teachers, times, locations.

To access this information, I need to log in to my university's secured website and search for individual classes. How do programs (i've seen iphone apps, etc), where you search for a university, a class and it displays CURRENT and updated rosters for classes.

How do programs access this data without an API, or login credentials to pull available course data.

Upvotes: 0

Views: 288

Answers (1)

dsw88
dsw88

Reputation: 4592

Assuming the university doesn't have some sort of API, these apps are probably just scraping html data off the screen and pulling relevant pieces out for usage in their app. This can work, but it's always a pain to work because sites will often change their HTML structure, which would necessitate you rewriting your screen scraper to compensate.

Be aware that certain university data usage can be restricted under federal FERPA laws. Since the university doesn't have a public API of some sort to get that data, you'd be wise to check with them on how they feel about you pulling data from their site for usage in your app. That'll avoid big problems down the line when they find out your app is grabbing data from their secured website.

Upvotes: 1

Related Questions