How do continuous integration and automated tests look in practice?

Question

I know what unit testing is, why it should be used and how to write them. However, I'm not sure do I understand the terms "automated tests" and "Continuous integration" correctly. I have some general understanding about those but would like to know how do they look in practice, for example in PHP (or at least in other languages). So:

1. Is the term "automated testing" the same as "unit testing"? Or "automated testing" should be considered as a collection of unit test classes?

2. "Continuous integration" is just a matter of organizing the work in dev team and its goal is to have a most-recent AND tested software revision at the end of each day. So, everybody should put his code into codebase frequently (for example at the end of every working day), and also, everybody should commit they unit tests into some test manager that runs all unit tests (performs automated testing). So, after automated testing (execution of all unit tests) and hot-fixing bugs at the end of day, we have a shippable software code revision.

Answer 1

Unit Testing != Automated testing

Unit testing is pretty old. It's verifying (tiny) units (methods, classes) of your code work as expected. e.g. Verifying boundary conditions, logical paths, etc. Thus unit-testing can be manual as well.. e.g. you key in some inputs and manually verify that it works.
Automated testing is tests that don't require a human to execute the tests or interpret the results. So these are tests that can be run on command and will tell you if everything works as expected or not. Automated tests could be automated unit tests or they could be system level / acceptance tests or performance tests or anything else...

CI

You're pretty close... except that it isn't at the end of the day, it's almost all the time. The main purpose of a CI build is quick feedback (Bonus: a potentially shippable revision of the software at all times) and less time integrating everyone's changes. Whenever someone makes a check-in, the CI mechanism would check out all the source, build to check for compilation errors and then most probably kick off unit tests and finally some system level tests. This way, a bad check-in / failures are detected as early as possible (and in some cases, you configure it such that the check-in never gets into version control i.e. reverted).

Answer 2

Automated tests vs. unit tests: my understanding is that unit tests are a sub-category of automated tests. As the name suggests, automated tests are tests that do not require human intervention to run: a machine should be able to run them. Unit tests are a subset of them: they verify a single unit of functionality. The best definition I saw for them was by Kent Beck: "if a test fails, how many things could be wrong? the closer the answer is to 1, the more "unit-y" the test is." Other automated tests could be integration tests, which will tell you that the app doesn't behave altogether as expected, but doesn't necessarily tell you which piece of the app is defective.

Continuous integration is about limiting "integration hell", aka "but it works on my machine!". Instead of hoping that multiple developers always make sure that their changes stay in sync with the whole team effort and that they didn't break the application, continuous integration strives to have a "defensive" repository mechanism, that is, every time someone commit, the repository makes sure that nothing is broken. At a minimum, this involves making sure that the latest commit still builds, at best, it also enforces that existing functionality is still there, by running automated tests.

Answer 3

I'm still getting used to the terms myself, but my understanding of CI and automated testing is that as you push code, tests are being run.

We use Jenkins with HipChat integration, so every time someone pushes to GitHub, Jenkins runs the tests and lets us know, sometimes loudly, if the build has been broken.