CODEWITHSUNDEEP
c++artificial-intelligencegemma
Chris Olson
Chris Olson

Reputation: 1

Gemma 2 gemma.cpp config.h parameters for Gemma27B

In the gemma.cpp source distribution there is a file that controls the model used (and a lot of other files that map from text to the model). I've tracked everything down and fixed all of the text mappings. I just need to figure out the entry in the config.h file.

I tried the following:

struct ConfigGemma27B {
    static constexpr int kSeqLen = gcpp::kSeqLen;
    static constexpr int kVocabSize = 256128;
    static constexpr int kLayers = 46;
    static constexpr int kModelDim = 4608;
    static constexpr int kFFHiddenDim = 16 * 4608 / 2;  // = 24576
    static constexpr int kHeads = 32;
    static constexpr int kKVHeads = 16;  // standard MHA
    static constexpr int kQKVDim = 256;  // query size == key size == value size
    static constexpr int kTopK = gcpp::kTopK;
};

based on:

struct ConfigGemma7B {
  static constexpr int kSeqLen = gcpp::kSeqLen;
  static constexpr int kVocabSize = 256000;
  static constexpr int kLayers = 28;
  static constexpr int kModelDim = 3072;
  static constexpr int kFFHiddenDim = 16 * 3072 / 2;  // = 24576
  static constexpr int kHeads = 16;
  static constexpr int kKVHeads = 16;  // standard MHA
  static constexpr int kQKVDim = 256;  // query size == key size == value size
  static constexpr int kTopK = gcpp::kTopK;
};

and the paper, here: https://storage.googleapis.com/deepmind-media/gemma/gemma-2-report.pdf

I ran it, and it never returns. My question is, did I just not wait long enough (it is a HUGE model and consumes a vast amount of memory) or did I get one of the configurations wrong?

Upvotes: 0

Views: 53

Answers (0)

Related Questions