std::unordered_map::find using a type different than the Key type?

Question

I have an unordered_map that uses a string-type as a key:

std::unordered_map<string, value> map;

A std::hash specialization is provided for string, as well as a suitable operator==.

Now I also have a "string view" class, which is a weak pointer into an existing string, avoiding heap allocations:

class string_view {
    string *data;
    size_t begin, len;
    // ...
};  

Now I'd like to be able to check if a key exists in the map using a string_view object. Unfortunately, std::unordered_map::find takes a Key argument, not a generic T argument.

(Sure, I can "promote" one to a string, but that causes an allocation I'd like to avoid.)

What I would've liked instead was something like

template<class Key, class Value>
class unordered_map
{
    template<class T> iterator find(const T &t);
};

which would require operator==(T, Key) and std::hash<T>() to be suitably defined, and would return an iterator to a matching value.

Is there any workaround?

Answer 1

boost::unordered_map has transparent lookup support. You just need to specify transparent hash and transparent eq operations

#include <string>
#include <boost/unordered/unordered_map.hpp>

struct transparent_hash
{
    using is_transparent = void;
    template<class T>
    std::size_t operator()(const T& x) const
    {
        return std::hash<T> {}(x);
    }
};


// Same as c++14's std::equal_to<void>  
struct transparent_equal  
{
    using is_transparent = void;
    template<class T, class U>
    bool operator()(const T& x, const U& y) const
    {
        return x == y;
    }
};

boost::unordered_map<string_view, std::string, transparent_hash, transparent_equal> m;

Answer 2

P0919R2 Heterogeneous lookup for unordered containers has been merged in the C++2a's working draft!

The abstract seems a perfect match w.r.t. my original question :-)

Abstract

This proposal adds heterogeneous lookup support to the unordered associative containers in the C++ Standard Library. As a result, a creation of a temporary key object is not needed when different (but compatible) type is provided as a key to the member function. This also makes unordered and regular associative container interfaces and functionality more compatible with each other.

With the changes proposed by this paper the following code will work without any additional performance hits:

template<typename Key, typename Value>
using h_str_umap = std::unordered_map<Key, Value, string_hash>;
h_str_umap<std::string, int> map = /* ... */;
map.find("This does not create a temporary std::string object :-)"sv);

Full example from cppreference

#include <cstddef>
#include <functional>
#include <iostream>
#include <string>
#include <string_view>
#include <unordered_map>
 
using namespace std::literals;
 
struct string_hash
{
    using hash_type = std::hash<std::string_view>;
    using is_transparent = void;
 
    std::size_t operator()(const char* str) const        { return hash_type{}(str); }
    std::size_t operator()(std::string_view str) const   { return hash_type{}(str); }
    std::size_t operator()(std::string const& str) const { return hash_type{}(str); }
};
 
int main()
{
    // simple comparison demo
    std::unordered_map<int,char> example = {{1, 'a'}, {2, 'b'}};
 
    if (auto search = example.find(2); search != example.end())
        std::cout << "Found " << search->first << " " << search->second << '\n';
    else
        std::cout << "Not found\n";
 
    // C++20 demo: Heterogeneous lookup for unordered containers (transparent hashing)
    std::unordered_map<std::string, size_t, string_hash, std::equal_to<>> map{{"one"s, 1}};
    std::cout << std::boolalpha
        << (map.find("one")   != map.end()) << '\n'
        << (map.find("one"s)  != map.end()) << '\n'
        << (map.find("one"sv) != map.end()) << '\n';
}

Answer 3

I faced an equal problem.

We need two structs:

struct string_equal {
    using is_transparent = std::true_type ;

    bool operator()(std::string_view l, std::string_view r) const noexcept
    {
        return l == r;
    }
};


struct string_hash {
    using is_transparent = std::true_type ;

    auto operator()(std::string_view str) const noexcept {
        return std::hash<std::string_view>()(str);
    }
};

For unordered_map:

template <typename Value>
using string_unorderd_map = std::unordered_map<std::string, Value, string_hash, string_equal>;

For unordered_set:

using string_unorderd_set = std::unordered_set<std::string, string_hash, string_equal>;

Now using string_view is possible.

Answer 4

I'll just present one variation I found on github, it involves defining a new map class that wraps the std.
Redefining some key API to intercept the adaptors we want, and use a static string to copy the key.
It's not necessary a good solution, but it's interesting to know it exists for people who deems it enough.

original:
https://gist.github.com/facontidavide/95f20c28df8ec91729f9d8ab01e7d2df

code gist:

template <typename Value>
class StringMap: public std::unordered_map<std::string, Value>
{
public:
    typename std::unordered_map<string,Value>::iterator find(const nonstd::string_view& v )
    {
        tmp_.reserve(  v.size() );
        tmp_.assign( v.data(), v.size() );
        return std::unordered_map<string, Value>::find(tmp_);
    }

    typename std::unordered_map<std::string,Value>::iterator find(const std::string& v )
    {
        return std::unordered_map<std::string, Value>::find(v);
    }

    typename std::unordered_map<std::string,Value>::iterator find(const char* v )
    {
        tmp_.assign(v);
        return std::unordered_map<std::string, Value>::find(v);
    }

private:
    thread_local static std::string tmp_;
};

credits:
Davide Faconti

Answer 5

Sorry for answering this very old question, but it still comes up in search engine results... In this case your unordered_map is using the string type as its key, the find method is looking for a reference to a string which will not generate an allocation. Your string_view class stores a pointer to a string. Therefore your string_view class can dereference the pointer into a ref of the type needed for your map without causing an allocation. The method would look like this...

string &string_view::getRef() const
{
    return *_ptr;
}

and to use the string_view with the map it would look like this

auto found=map.find(string_view_inst.getRef());

note that this will not work for the c++17 string_view class as it does not internally store a std::string object

ps. Your string_view class is probably not great for cpu caches as it stores a pointer to a string allocated somewhere on the heap, and the string itself stores a pointer to the actual data located somewhere else on the heap. Every time you access your string_view it will result in a double dereference.

Answer 6

Yet another option is to split the lookup and the data management by using multiple containters:

std::unordered_map<string_view, value> map;
std::vector<unique_ptr<const char[]>> mapKeyStore;

Lookups are done using string_views without the need of allocations. Whenever a new key is inserted we need to add a real string allocation first:

mapKeyStore.push_back(conv(str)); // str can be string_view, char*, string... as long as it converts to unique_ptr<const char[]> or whatever type
map.emplace(mapKeyStore.back().get(), value)

It would be much more intuitive to use std::string in the mapKeyStore. However, using std::string does not guarantee unchanging string memory (e.g. if the vector resizes). With the unique_ptr this is enforced. However, we need some special conversion/allocation routine, called conv in the example. If you have a custom string container which guarantees data consistency under moves (and forces the vector to use moves), then you can use it here.

The drawback

The disadvantage of the above method is that handling deletions is non-trivial and expensive if done naive. If the map is only created once or only growing this is a non-issue and the above pattern works quite well.

Running example

The below example includes a naive deletion of one key.

#include <vector>
#include <unordered_map>
#include <string>
#include <string_view>
#include <iostream>
#include <memory>
#include <algorithm>

using namespace std;
using PayLoad = int;

unique_ptr<const char[]> conv(string_view str) {
    unique_ptr<char[]> p (new char [str.size()+1]);
    memcpy(p.get(), str.data(), str.size()+1);
    return move(p);
}

int main() {
    unordered_map<string_view, PayLoad> map;
    vector<unique_ptr<const char[]>> mapKeyStore;
    // Add multiple values
    mapKeyStore.push_back(conv("a"));
    map.emplace(mapKeyStore.back().get(), 3);
    mapKeyStore.push_back(conv("b"));
    map.emplace(mapKeyStore.back().get(), 1);
    mapKeyStore.push_back(conv("c"));
    map.emplace(mapKeyStore.back().get(), 4);
    // Search all keys
    cout << map.find("a")->second;
    cout << map.find("b")->second;
    cout << map.find("c")->second;
    // Delete the "a" key
    map.erase("a");
    mapKeyStore.erase(remove_if(mapKeyStore.begin(), mapKeyStore.end(),
        [](const auto& a){ return strcmp(a.get(), "a") == 0; }),
        mapKeyStore.end());
    // Test if verything is OK.
    cout << '\n';
    for(auto it : map)
        cout << it.first << ": " << it.second << "\n";

    return 0;
}

Of course, the two containers can be put into a wrapper which handles the insertion and deletion for its own.

Answer 7

This solution has drawbacks, which may or may not make it unviable for your context.

You can make a wrapper class:

struct str_wrapper {
  const char* start, end;
};

And change your map to use str_wrapper as its key. You'd have to add 2 constructors to str_wrapper, one for std::string and one for your string_view. The major decision is whether to make these constructors perform deep or shallow copies.

For example, if you use std::string only for inserts and str_view only for lookups, you'd make the std::string constructor deep and the str_view one shallow (this can be enforced at compile time if you use a custom wrapper around unordered_map). If you care to avoid memory leaks on the deep copy you would need additional fields to support proper destruction.

If your usage is more varied, (looking up std::string's or inserting by str_view), there will be drawbacks, which again, might make the approach too distasteful so as to be unviable. It depends on your intended usage.

Answer 8

As mentioned above, C++14 does not provide heterogeneous lookup for std::unordered_map (unlike std::map). You can use Boost.MultiIndex to define a fairly close substitute for std::unordered_map that allows you to look up string_views without allocating temporary std::strings:

Live Coliru Demo

#include <boost/multi_index_container.hpp>
#include <boost/multi_index/hashed_index.hpp>
#include <boost/multi_index/member.hpp>
#include <string>

using namespace boost::multi_index;

struct string_view
{
  std::string *data;
  std::size_t begin,len;
};

template<typename T,typename Q>
struct mutable_pair
{
  T         first;
  mutable Q second;
};

struct string_view_hash
{
  std::size_t operator()(const string_view& v)const
  {
     return boost::hash_range(
       v.data->begin()+v.begin,v.data->begin()+v.begin+v.len);
  }
  std::size_t operator()(const std::string& s)const
  {
     return boost::hash_range(s.begin(),s.end());
  }
};

struct string_view_equal_to
{
  std::size_t operator()(const std::string& s1,const std::string& s2)const
  {
     return s1==s2;
  }
  std::size_t operator()(const std::string& s1,const string_view& v2)const
  {
     return s1.size()==v2.len&&
            std::equal(
              s1.begin(),s1.end(),
              v2.data->begin()+v2.begin);
  }
  std::size_t operator()(const string_view& v1,const std::string& s2)const
  {
     return v1.len==s2.size()&&
            std::equal(
              v1.data->begin()+v1.begin,v1.data->begin()+v1.begin+v1.len,
              s2.begin());
  }
};

template<typename Q>
using unordered_string_map=multi_index_container<
  mutable_pair<std::string,Q>,
  indexed_by<
    hashed_unique<
      member<
        mutable_pair<std::string,Q>,
        std::string,
        &mutable_pair<std::string,Q>::first
      >,
      string_view_hash,
      string_view_equal_to
    >
  >
>;

#include <iostream>

int main()
{
  unordered_string_map<int> m={{"hello",0},{"boost",1},{"bye",2}};

  std::string str="helloboost";
  auto it=m.find(string_view{&str,5,5});
  std::cout<<it->first<<","<<it->second<<"\n";
}

Output

boost,1

Answer 9

It looks like only as recently as C++14 did even the basic map get such a templated find for is_transparent types in the comparison. Most likely the correct implementation for hashed containers was not immediately evident.

As far as I can see your two options are:

• Just do the allocation and profile to see if maybe it's not actually a problem.
• Take a look at boost::multi_index (http://www.boost.org/doc/libs/1_60_0/libs/multi_index/doc/index.html) and have both string and string_view indexes into the container.

Answer 10

You could allow your view to be implicitly convertible to a std::string:

class StringView {
    // ...
    operator std::string() const
    {
        return data->substr(begin, len);
    }
    // ...
};