How to integrate multiple components into an API

Question

I am working on tying together multiple components that I have developed for my undergraduate thesis, and have hit a road block. There are five components to my project:

• Sqlite Database (Considering switching to postgres to make propping it up on Amazon RDS easier)
• Python Data Scraper
• C++ Data Transformation/Computation Layer
• C++ Data Output Component
• Angular Front-End (To be developed)

Currently all of my components are duct-taped together with a bash script, with the Angular component non-existent.

What I want to do is the following:

• Make an AWS hosted API which does the following:

  • Make the Data Scraper / Data Transformation layer into a cron job
  • Make the Data Output component be callable by the Angular site for certain values (at the moment I only have 1 API call in mind)

Problems:

• Do not know how to integrate components with various languages into an API
• Do not know how (or if it is even possible) to prop up a SQlite db onto some server.

Any suggestions to simplify this would be greatly appreciated. I have had these components completed for about a month but have not been able to figure out how to tie them together.

Thanks!

Answer 1

You make things very complicated :-)

You have several languages at play here, which makes things much trickier. I would consider rewriting your C++ code in python, or making python wrappers around it: https://docs.python.org/2/extending/extending.html so that you can use a python framework like one of these: http://blog.miguelgrinberg.com/post/designing-a-restful-api-with-python-and-flask or http://python-eve.org.

If you want to use RDS, you will have to switch to one of their supported databases. If you are OK running your own database, you will already be using a server to run your code on, just use an EBS volume there (don't use ephemeral disk unless you are going to do a very robust backup/replication process). But considering what you have built, I would really consider using something with less maintenance overhead. If you are willing to pay for it, Aurora just got released for public use, and removes most of the db administration overhead work.