CODEWITHSUNDEEP
continuous-integration
Lethargy
Lethargy

Reputation: 1889

Continuous Integration testing with physical devices

I've been wondering how you go about doing CI-style testing when you're dealing with physical devices.

I imagine you have a suite of tests, and a pool of devices against which they can be run. Additionally:

What CI servers have support for this?

I'm still interested in those which have partial support, either natively or through plugins, as I'm interested in how it's done.

Upvotes: 2

Views: 819

Answers (3)

snakehiss
snakehiss

Reputation: 8774

I wouldn't get too caught up in what a CI system is supposed to do or not do. Instead I would focus on the problem you are trying to solve. It sounds like that problem is to facilitate development on multiple platforms. You can use the concept of Continuous Integration and add to it successfully address the issue. I know, because I've done it in the past.

I implemented a build system for code that needed to compile and test successfully on 4 different platforms (nt, wince, linux-arm, linux-x86). The CI server would:

  • Used a linux and winnt build server compilation (and cross compilation)
  • The compiled tests and supporting libs would then be copied to the appropriate devices and an automated test run executed.
  • After the test suite was completed the log would be copied back, (or it was written to a network mounted fs)
  • If the test suite was successful we would tag the source, and package the libs and executables.

This same platform was reused for developer verification before commits. Developers would run a partial build and test (only updated source would be recompiled and those tests rerun). The CI would execute a full build (from scratch).

Our build were pretty fast because we had a proper DAG for build dependencies. This allowed for concurrent compilation within a platform build. Each platform build was also concurrent. As a result partial builds took a few seconds, full builds took ~30 minutes. Our build servers were quite beefy (optimized for fast compiles) and the codebase was of moderate size (I don't remember the stats).

Upvotes: 1

andreadi
andreadi

Reputation: 1965

I can suggest Test Manager that is part of the TFS suite of Microsoft. I have not tried it with many different environments apart from windows based though I know there are many connectors. For windows based environments I believe it will satisfy most needs.

I use it for nightly builds to perform smoke tests (Turn it on, see if any smoke comes out) but you have to be careful to keep tests small in order to have them finished in a matter of hours and not days, if you want it to be part of your CI.

Then when you have a good enough quality you can proceed onto regression tests and integration tests if needed.

Upvotes: 1

Prisoner ZERO
Prisoner ZERO

Reputation: 14176

Continuous Integration enables a team integrate & test their work frequently. Automated builds are meant to compile, link and run unit-tests. You want your CI to run fast, especially if you run it at every check-in. That is why you would want to restrict CI activities to simple confirmation of the build and unit tests alone. What you're asking seems more-along the lines of quality assurance (QA) testing...and having QA failures mixed into your CI efforts would detract development efforts from progressing.

As such, I'm more under the impression activities associated with CI are not dependent upon the final physical machine said work may eventually be migrated to.

Now...this doesn't mean you CAN'T take the CI-compiled package and run it against some final target-machine....but again...that is really considered a seperate activity.

This seems to be re-enforced in the following article by Martin Fowler.

Notice he doesn't talk about the final targeted devices...only the build machine.

Upvotes: 1

Related Questions