How do you decide whether to use a library or write your own implementation

Question

Inspired by this question which started out innocently but is turning into a major flame war.

Let's say you need to a utility method - reasonably straightforward but not a one-liner. Quoted question was how to repeat a string X times. How do you decide whether to use a 3rd party implementation or write your own?

The obvious downside to 3rd party approach is you're adding a dependency to your code. But if you're writing your own you need to code it, test it, (maybe) profile it so you'll likely end up spending more time.

I know the decision itself is subjective, but criteria you use to arrive at it should not be.

So, what criteria do you use to decide when to write your own code?

Answer 1

Another consideration is security.

If a black-hat hacker finds a bug in your code they can create an exploit and sell it for money. The more popular the library is, the more the exploit worth. Think about OpenSSL or Wordpress exploits. If you re-implement the code, chances that your code is not vulnerable exactly the same way the popular library is. And if your lib is not popular, then an zero-day exploit of your code probably wouldn't worth much, and there is a good chance your code is not targeted by bounty hunters.

Another consideration is language safety. C language can be very fast. But from the security standpoint it's asking for trouble. If you reimplement the lib in some script language, chances of arbitrary code execution exploits are low (as long as you know the possible attack vectors, like serialization, or evals).

Answer 2

Keep it in balance

You should keep several criteria in balance. I'd consider a few topics and ask a few questions.

Developing time VS maintenance time

Can I develop what I need in a few hours? If yes, why do I need a library? If I get a lib am I sure that it will not cause hours spent to debug and documentation reading? The answer - if I need something obvious and straightforward I don't need an extra-flexible lib.

Simplicity VS flexibility

If I need just an error wrapper do I need a lib with flexible types and stack tracing and color prints and.... Nope! Using even beautifully designed but flexible and multipurpose libs could slow your code. If you plan to use 2% of functionality you don't need it.

Dig task VS small task

Did I faced a huge task and I need external code to solve it? Definitely AMQP or SQL operations is too big tasks to develop from scratch but tiny logging could be solved in place. Don't use external libs to solve small tasks.

My own libs VS external libs

Sometimes is better to grow your own library because it is for 100% used, for 100% appropriate your goals, you know it best, it is always up to date with your applications. Don't build your own lib just to be cool and keep in mind that a lot of libs in your vendor directory developed "just to be cool".

Answer 3

General Decision

Before deciding on what to use, I will create a list of criteria that must be met by the library. This could include size, simplicity, integration points, speed, problem complexity, dependencies, external constraints, and license. Depending on the situation the factors involved in making the decision will differ.

Generally, I will hunt for a suitable library that solves the problem before writing my own implementation. If I have to write my own, I will read up on appropriate algorithms and seek ideas from other implementations (e.g., in a different language).

If, after all the aspects described below, I can find no suitable library or source code, and I have searched (and asked on suitable forums), then I will develop my own implementation.

Complexity

If the task is relatively simple (e.g., a MultiValueMap class), then:

1. Find an existing open-source implementation.
2. Integrate the code.
3. Rewrite it, or trim it down, if it excessive.

If the task is complex (e.g., a flexible object-oriented graphing library), then:

1. Find an open-source implementation that compiles (out-of-the-box).
2. Execute its "Hello, world!" equivalent.
3. Perform any other evaluations as required.
4. Determine its suitability based on the problem domain criteria.

Speed

If the library is too slow, then:

1. Profile it.
2. Optimize it.
3. Contribute the results back to the community.

If the code is too complex to be optimized, and speed is a factor, discuss it with the community and provide profiling details. Otherwise, look for an equivalent, but faster (possibly less feature-rich) library.

API

If the API is not simple, then:

• Write a facade and contribute it back to the community.
• Or find a simpler API.

Size

If the compiled library is too large, then:

• Compile only the necessary source files.
• Or find a smaller library.

Bugs

If the library does not compile out of the box, seek alternatives.

Dependencies

If the library depends on scores of external libraries, seek alternatives.

Documentation

If there is insufficient documentation (e.g., user manuals, installation guides, examples, source code comments), seek alternatives.

Time Constraints

If there is ample time to find an optimal solution, then do so. Often there is not sufficient time to write from scratch. And usually there are a number of similar libraries to evaluate. Keep in mind that, by meticulous loose coupling, you can always swap one library for another. Find what works, initially, and if it later becomes a burden, replace it.

Development Environment

If the library is tied to a specific development environment, seek alternatives.

License

Open source.

Answer 4

If the functionality is only a small part of the app, or if your needs are the same as everyone else's, then a library is probably the way to go. If you need to consume and output JSON, for example, you can probably knock something together in five minutes to handle your immediate needs. But then you start adding to it, bit by bit. Eventually, you have all the functionality that you would find in any library, but 1) you had to write it yourself and 2) it isn't a robust and well document as what you would find in a library.

If the functionality is a big part of the app, and if your needs aren't exactly the same as everyone else's, then think much more carefully. For example, if you are doing machine learning, you might consider using a package like Weka or Mahout, but these are two very different beasts, and this component is likely to be a significant part of your application. A library in this case could be a hindrance, because your needs might not fit the design parameters of the original authors, and if you attempt to modify it, you will need to worry about a much larger and more complex system than the minimum that you would build yourself.

There's a good article out there talking about sanitizing HTML, and how it was a big part of the app, and something that would need to be heavily tuned, so using an outside library wasn't the best solution, in spite of the fact that there were many libraries out that did exactly what seemed to be called for.

Answer 5

10 questions ...

+++ (use library) ... --- (write own library)

1. Is the library exactly what I need? Customizable in a few steps? +++
2. Does it provide almost all functionality? Easily extensible? +++
3. No time? +++
4. It's good for one half and plays well with other? ++
5. Hard to extend, but excellent documentation? ++
6. Hard to extend, yet most of the functionality? +
7. Functionality ok, but outdated? -
8. Functionality ok, .. but weird (crazy interface, not robust, ...)? --
9. Library works, but the person who needs to decide is in the state of hybris? ---
10. Library works, manageable code size, portfolio needs update? ---

Some thoughts ...

If it is something that is small but useful, probably for others, too, then why now write a library and put it on the web. The cost publishing this kind of small libraries decreased, as well as the hurdle for others to tune in (see bitbucket or github). So what's the criteria?

Maybe it should not exactly replicate an existing already known library. If it replicates something existing, it should approach the problem from new angle, or better it should provide a shorter or more condensed* solution.

*_/fun

Answer 6

For me this would be a fairly easy answer.

If you need to be cost effective, then it would probably be best to try and find a library/framework that does what you want. If you can't find it, then you will be forced to write it or find a different approach.

If you have the time and find it fun, write one. You will learn a lot along the way and you can give back to the open source community with you killer new bundle of code. If you don't, well, then don't. But if you can't find one, then you have to write it anyway ;)

Personally, if I can justify writing a library, I always opt for that. It's fun, you learn a lot about what you are directing your focus towards, and you have another tool to add to your arsenal and put on your CV.

Answer 7

If it's a trivial function, it's not worth pulling in an entire library.

If it's a non-trivial function, then it may be worth it.

If it's multiple functions which can all be handled by pulling in a single library, it's almost definitely worth it.