http/2 dynamic table decoding goes wrong

Question

I try to implement http/2 code, i'm trying to send request to the google. I looked in wireshark how the Mozilla Firefox does it. Here is an example:

Here is the first request to google, after a first SETTINGS frame

Header Block Fragment: 8205a563fc9a52c5649012cf829a03f1f3372e66f78c9eb6...
[Header Length: 590]
[Header Count: 11]
Header: :method: GET
    Representation: Indexed Header Field

Header: :path: /?gfe_rd=cr&ei=oVxgWYizHcyRZK-lqKgG&gws_rd=ssl
    Representation: Literal Header Field without Indexing - Indexed Name

Header: :authority: www.google.ru
    Representation: Literal Header Field with Incremental Indexing - Indexed Name

Header: :scheme: https
    Representation: Indexed Header Field

Header: user-agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:54.0) Gecko/20100101 Firefox/54.0
    Representation: Literal Header Field with Incremental Indexing - Indexed Name

Header: accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
    Representation: Literal Header Field with Incremental Indexing - Indexed Name

Header: accept-language: ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3
    Representation: Literal Header Field with Incremental Indexing - Indexed Name

Header: accept-encoding: gzip, deflate, br
    Representation: Literal Header Field with Incremental Indexing - Indexed Name

Header: cookie: NID=107=L67kU0sEsfzwpItJcUZyRfjKENJ1xa2N8_m4xqnVLCj9dW289EDyzg7ZvXZCt9kEII0nd5BA2Tns95vlkCGlGP850ULPoOKFeMJoaQT_0o7Cl3skG4PDdQgUW2hlRh7Z
    Representation: Literal Header Field with Incremental Indexing - Indexed Name

Header: dnt: 1
    Representation: Literal Header Field with Incremental Indexing - New Name

Header: upgrade-insecure-requests: 1
    Representation: Literal Header Field with Incremental Indexing - New Name
Padding: <MISSING>

So the dynamic table for client will look like this, assuming that i understood this right:

1 ->upgrade-insecure-requests: 1
2 ->dnt :1
3 ->cookie: NID=107=kjxoLmF8StT6CO8a8T-z68DAIJB6cyzAwqWOjjmXGubtu2pmJSfGPCjUAo9D-OTzuk03u34YLxLMjH236FBDzKVYayleievvTkamyhrLuOz4AYT_7KiuVIS5JA_BuVEa
4 ->accept-encoding: gzip, deflate, br
5 ->accept-language: ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3
6 ->accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
7 ->user-agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:54.0) Gecko/20100101 Firefox/54.0
8 ->:authority: www.google.ru

All the headers marked as Representation: Literal Header Field with Incremental Indexing are inserted in the dynamic table, those who were inserted first, move to the higher index, and the new entries are on top.

But this table doesn't work, because when browser does the second request, to download some images (google logo or whatever), some mess is hapening,

second request decoded:

[Header Length: 535]
[Header Count: 11]
Header: :method: GET
    Representation: Indexed Header Field

Header: :path: /images/hpp/ic_wahlberg_product_core_48.png8.png
    Representation: Literal Header Field without Indexing - Indexed Name

Header: :authority: www.google.ru
    Representation: Indexed Header Field

Header: :scheme: https
    Representation: Indexed Header Field

Header: user-agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:54.0) Gecko/20100101 Firefox/54.0
    Representation: Indexed Header Field

Header: accept: */*
    Representation: Literal Header Field with Incremental Indexing - Indexed Name

Header: accept-language: ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3
    Representation: Indexed Header Field

Header: accept-encoding: gzip, deflate, br
    Representation: Indexed Header Field

Header: referer: https://www.google.ru/
    Representation: Literal Header Field with Incremental Indexing - Indexed Name

Header: cookie: NID=107=j8x_CchswQznRlhYWeTtf7rDE7QaxZBxbGeQevst16TVFJ3qOJjlsVwxEobdZaVAeUFboCuuUhu0VdJwujroT_u5hoGttx5Mcu4xnMZQIfK7OSMhqtSbO135AL8GlZWU
    Representation: Literal Header Field with Incremental Indexing - Indexed Name

Header: dnt: 1
    Representation: Indexed Header Field

Padding: <MISSING>

And here is the second request encoded:

82 = :method: GET ( index 2 in the static table is "method: GET" )
05 a3 60 d4 8e 62 a1 89 eb ad 83 12 2f 03 9e 88
cb 64 d1 57 61 e4 b4 89 88 87 b0 b1 34 f2 f5 d5
33 cb d7 54 df  = :path: /images/hpp/ic_wahlberg_product_core_48.png8.png (05 is the index of "path:" in static table, rest is huffman encoded literal )
c5 = :authority: www.google.ru ( index 8 at the dynamic table, 0xc5 & 0x7f = 69 (dec) static table length (61)? is this correct?
87 = scheme: https - static table
c4 = user-agent: Mozilla/5.0... ( index 7 at the dynamic table, 0xc5 & 0x7f = 68 (dec) static table length (61)? is this correct?

And here is where i can't understand it:

7f 04 83 f9 63 e7 - this suppose to mean accept: */* according to wireshark and it's signed as Literal Header Field with Incremental Indexing - Indexed Name, so the 0x7f shoud be the index of the header. The mask for literal incremental indexed is 0x40, so we do boolean AND on 0x7f with 0x3f, and we get ‭63‬ decimal, which is the dnt: 1 header from our dynamic table, this is competely incorrect. Can someone help, how does this decoded correctly? Does http/2 uses the same dynamic table for response decoding? ( maybe that's why my decoding is messed up ? )

Answer 1

You are calculating the index number wrong by only looking at a single byte (0x7f or 0x3f after stripping of the index field information). However you need to additionally look at how integers (and thereby also header field indices) are represented in HPACK. This is covered by section 5.1 Integer Representation in the HPACK specification.

You can see there that if all bits that are used for a number in the first byte are set to zero, you need to take the next byte into account too. This is happening in that case, since 0x3f means all 6 bits that are used are set to 1. So the actual index number is encoded as 0x7f 0x04. I think that should result in 63+4 = 67, which is the 6th entry of the dynamic table. Which would match your expected layout of the dynamic table after the first request.

Btw: It's interesting that it doesn't select index 19, which is accept from the static table. From my point of view there's no extra value to refer to the dynamic table here, when only the field name (and no value is taken), and looking things up in the static table might be less overhead for some implementations.

And to clarify this questions:

Does http/2 uses the same dynamic table for response decoding?

No, there is one instance of the dynamic table for all sent headers (request headers on client side) and one instance for all received headers (response headers on client side).