CODEWITHSUNDEEP
c++stringvisual-studio-2010memory
Fabian
Fabian

Reputation: 4211

Can I make std::string use less memory?

Before I begin, I need to state that my application uses lots of strings, which are on average quite small, and which do not change once created.

In Visual Studio 2010, I noticed that the capacity of std::string is at least 30. Even if I write std::string str = "test";, the capacity of str is 30. The function str.shrink_to_fit() does nothing about this although a function with the same name exists for std::vector and works as expected, namely decreasing the capacity so that capacity == size.

  1. Why does std::string::shrink_to_fit() not work at expected?
  2. How can I ensure that the string allocates the least amount of memory?

Upvotes: 7

Views: 2400

Answers (4)

Ilya Popov
Ilya Popov

Reputation: 4010

What you observe is a result of SSO (short string optimization), as pointed out by others.

What you could do about it depends on the usage pattern:

  • If you strings are parts of one big string, which is typical for parsing, you can use classes like std::experimental::string_view, GSL string_span, Google's StringPiece, LLVM's StringRef etc. which do not store data themselves but only refer to a piece of some other string, while providing interface similar to std::string.

  • If there are multiple copies of the same strings (especially long ones), it may make sense to use CoW (copy-on-write) strings, where copies share the same buffer using reference counter mechanism until modified. (But be aware of downsides)

  • If the strings are very short (just a few chars) it may make sense to write your own specialized class, something in line with Handling short codes by Andrzej

Whatever case you chose, it is important to establish good benchmarking procedure to clearly see what effect (if any) you get.

Upd: after rereading the introduction to the question, I think the third approach is the best for you.

Upvotes: 1

hungptit
hungptit

Reputation: 1444

If you are using a lot of small strings in your application then you might want to take a look at fbstring (https://github.com/facebook/folly/blob/master/folly/docs/FBString.md).

Upvotes: 0

NathanOliver
NathanOliver

Reputation: 180415

One reason that std::string::shrink_to_fit() does nothing is that it is not required to by the standard

Remarks: shrink_to_fit is a non-binding request to reduce capacity() to size(). [ Note: The request is non-binding to allow latitude for implementation-specific optimizations. —end note ]

If you want to make sure the string shrinks then you can use the swap() trick like

std::string(string_to_shrink).swap(string_to_shrink)

Another reason this may not work is that the implementer of std::string is allowed to implement short string optimization so you could always have a minimum size of 30 on your implementation.

Upvotes: 1

pmr
pmr

Reputation: 59811

  1. Your std::string implementation most likely uses some form of the short string optimization resulting in a fixed size for smaller strings and no effect for shrink_to_fit. Note that shrink_to_fit is non-binding for the implementation, so this is actually conforming.
  2. You could use a vector<char> to get more precise memory management, but would loose some of the additional functionality of std::string. You could also write your own string wrapper which uses a vector internally.

Upvotes: 8

Related Questions