mongodb insert fails due to socket exception

Question

I am working on a Java project in Eclipse. I have a staging server and a live server. Those two also have their own mongodbs, which run on a different server on two different ports (29017 and 27017). Via a Junit Test I want to copy data from the live mongo to the devel mongo.

Weirdest thing: sometimes it works and sometimes I get a socket error. I wonder why mongo sometimes completely refuses to write inserts and on other days it works flawlessly. Here is an excerpt of the mongo log file (the one where code gets inserted) and the Junit test script:

mongo log:

Thu Mar 14 21:01:04 [initandlisten] connection accepted from xx.xxx.xxx.183:60848 #1    (1 connection now open)
Thu Mar 14 21:01:04 [conn1] run command admin.$cmd { isMaster: 1 }
Thu Mar 14 21:01:04 [conn1] command admin.$cmd command: { isMaster: 1 } ntoreturn:1 keyUpdates:0  reslen:90 0ms
Thu Mar 14 21:01:04 [conn1] opening db:  repgain
Thu Mar 14 21:01:04 [conn1] query repgain.editorconfigs query: { $and: [ { customer: "nokia" }, { category: "restaurant" } ] } ntoreturn:0 keyUpdates:0 locks(micros) W:5302 r:176 nreturned:0 reslen:20 0ms
Thu Mar 14 21:01:04 [conn1] Socket recv() errno:104 Connection reset by peer xx.xxx.xxx.183:60848
Thu Mar 14 21:01:04 [conn1] SocketException: remote: xx.xxx.xxx.183:60848 error: 9001 socket exception [1] server [xx.xxx.xxx.183:60848]
Thu Mar 14 21:01:04 [conn1] end connection xx.xxx.xxx.183:60848 (0 connections now open)

junit test script:

public class CopyEditorConfig {

protected final Log logger = LogFactory.getLog(getClass());

private static final String CUSTOMER = "customerx";
private static final String CATEGORY = "categoryx";

@Test
public void test() {
    try {

        ObjectMapper om = new ObjectMapper();

        // script copies the config from m2 to m1.

        Mongo m1 = new Mongo("xxx.xxx.com", 29017); // devel
        Mongo m2 = new Mongo("yyy.yyy.com", 27017); // live
        Assert.assertNotNull(m1);
        Assert.assertNotNull(m2);

        logger.info("try to connect to db \"dbname\"");

        DB db2 = m2.getDB("dbname");

        logger.info("get collection \"config\"");
        DBCollection c2 = db2.getCollection("config");
        JacksonDBCollection<EditorTabConfig, ObjectId> ec2 = JacksonDBCollection.wrap(c2, EditorTabConfig.class, ObjectId.class);

        logger.info("find entry with customer {" + CUSTOMER + "} and category {" + CATEGORY + "}");
        EditorTabConfig config2 = ec2.findOne(DBQuery.and(DBQuery.is("customer", CUSTOMER), DBQuery.is("category", CATEGORY)));

        // config
        if (config2 == null) {
            logger.info("no customer found to copy.");
        } else {
            logger.info("Found config with id: {" + config2.objectId + "}");
            config2.objectId = null;
            logger.info("copy config");
            boolean found = false;

            DB db1 = m1.getDB("dbname");

            DBCollection c1 = db1.getCollection("config");
            JacksonDBCollection<EditorTabConfig, ObjectId> ec1 = JacksonDBCollection.wrap(c1, EditorTabConfig.class, ObjectId.class);

            EditorTabConfig config1 = ec1.findOne(DBQuery.and(DBQuery.is("customer", CUSTOMER), DBQuery.is("category", CATEGORY)));

            if (config1 != null) {
                found = true;
            }

            if (found == false) {
                WriteResult<EditorTabConfig, ObjectId> result = ec1.insert(config2);
                ObjectId id = result.getSavedId();
                logger.info("INSERT config with id: " + id);
            } else {
                logger.info("UPDATE config with id: " + config1.objectId);
                ec1.updateById(config1.objectId, config2);
            }

            StringWriter sw = new StringWriter();
            om.writeValue(sw, config2);
            logger.info(sw);
        }

    } catch (Exception e) {
        logger.error("exception occured: ", e);
    }
}
}

Running this script seems like a success when I read the log in eclipse. I get an id for both c1 and c2 and the data is also here. The log even states, that it didn't find the config on devel and inserts it. That also is true, if I put it there manually. It gets "updated" then. But the mongo log stays the same. The socket exception occurs, and the data is never written to the db.

I am out of good ideas to debug this. If you could, I'd be glad to get some tips how to go from here. Also, if any information is missing, please tell me, I'd be glad to share.

Regards, Alex

Answer 1

hi i encountered a similar issue . try going to mongoDB atlas -> network access -> add my current IP address. the issue got resolved . hope this can save your time .

Answer 2

Comment out the line in your mongod.conf that binds the IP to 127.0.0.1. Usually, it is set to 127.0.0.1 by default. For Linux, this config file location should be be /etc/mongod.conf. Once you comment that out , it will receive connections from all interfaces. This fixed it for me as i was getting these socket exceptions as well.

Answer 3

It seems you have a connection issue with mongo server. Below ways may help you better diagnose the mongo servers:

1. Try to get more information from log files:

   $less /var/log/mongo/mongod.log

   or customized log files defined in mongod.conf

2. Try to use mongostat to monitor the server state:

   $ mongostat -u ADMIN_USER -p ADMIN_PASS

3. Try to use mongo cli to check server runing status:

   $ mongo admin -u ADMIN_USER -p ADMIN_PASS
   $ db.serverStatus()
   

   More useful commands is at: http://docs.mongodb.org/manual/reference/method/

Sometimes it may come across with Linux system configs. Try to tune Linux for more connections and limits, and it may help. To check current Linux limits, run:

$ ulimit -a

Below suggestions may be helpful:

Each connection is seen by Linux as an open file. The default maximum number of open file is 1024. To increase this limit:

1. modify /etc/security/limits.conf:

   root    soft    nofile     500000
   root    hard    nofile     512000
   root    soft    nproc      500000
   root    hard    nproc      512000
   

2. modify /etc/sysctl.conf

   fs.file-max=360000
   net.ipv4.ip_local_port_range=1024 65000