CODEWITHSUNDEEP
mysqlsqlgomariadb
ET2019
ET2019

Reputation: 21

sqlx write: broken pipe

I am trying to update a huge text into a MariaDB table but keep getting broken pipe errors. The column has a longtext data type and from my logs it looks like the data has a size of ~1374069 bytes.

I've tried updating the MaxConnLife to several different values like half of my wait_timeout from MariaDB which is set to 28800 -> db.SetConnMaxLifetime(14400)

This is what my DB conn looks like

    loggerAdapter := zerologadapter.New(zerolog.New(os.Stdout))
    db := sqldblogger.OpenDriver(dsn, &mysql.MySQLDriver{}, loggerAdapter /*, using_default_options*/) // db is STILL *sql.DB

    db.SetConnMaxLifetime(14400)


    sqlxdb := sqlx.NewDb(db, "MySQL")

error log

{"level":"info","conn_id":"66g4YSswso2QqsQE","duration":0.289406,"query":"UPDATE campaigns SET vendor_products=? WHERE id=?","stmt_id":"Mygo_MuAaIuGYwe2","time":1605747034,"message":"PrepareContext"} 
[mysql] 2020/11/19 00:50:34 packets.go:145: write tcp IP->IP: write: broken pipe 
{"level":"ERROR","args":["[{\"Id\":1,\"UserId\":1,\"PlanId\":1,\"GroupI (1374069 bytes truncated)",130],"conn_id":"66g4YSswso2QqsQE","duration":2.138179,"ERROR":"invalid connection","query":"UPDATE campaigns SET vendor_products=? WHERE id=?","stmt_id":"Mygo_MuAaIuGYwe2","time":1605747034,"message":"StmtExecContext"} 
{"level":"ERROR","conn_id":"66g4YSswso2QqsQE","duration":0.00295,"ERROR":"driver: bad connection","query":"UPDATE campaigns SET vendor_products=? WHERE id=?","stmt_id":"Mygo_MuAaIuGYwe2","time":1605747034,"message":"StmtClose"}

Versions I am using

github.com/go-sql-driver/mysql v1.4.1
go 1.13
github.com/jmoiron/sqlx v0.0.0-20181024163419-82935fac6c1a

Any idea how I can solve this? Or should I just think of another solution? Like possibly storing this text to S3 instead of SQL?

I've done some research but none of these helped. Some of the stuff I found.

Upvotes: 1

Views: 1361

Answers (1)

ET2019
ET2019

Reputation: 21

Increasing the max_allowed_packet answers my question.

Reference: https://mariadb.com/docs/reference/mdb/system-variables/max_allowed_packet/

Upvotes: 1

Related Questions