CODEWITHSUNDEEP
javaspringhibernatejpa
user5412293
user5412293

Reputation:

Memory Leak Springboot

I have an application that seems to be using too much memory. I have been trying to find the source for a while. But still no luck.

I have read several articles pointing at JPA as the culprit for some of the memory issues with Spring Boot. I only have a single repository so I cannot imagine it be the issue.

@Repository
public interface  WordRepository extends JpaRepository<Word, Long> {

    @Query("SELECT w FROM Word w WHERE w.value IN (:words)")
    List<Word> findAllIn(@Param("words") List<String> words);

    Word findFirstByValue(String value);

    @Transactional
    Long removeByValue(String value);

    Page<Word> findAllByCategory(Pageable pageable, String category);
}

I have another class which is a helper for dropping the table. I cannot do that with JPA (that I know of) so I am getting a hold of the persistance object and using it to truncate the table.

@Service
public class WordRepositoryHelper {

    @PersistenceContext(unitName = "default")
    private EntityManager entityManager;

    @Transactional
    public int truncateWordTable() {
        final String sql = "truncate table word";
        entityManager.createNativeQuery(sql).executeUpdate();
        return 1;
    }
}

I am using them as in here.

@Service
@SuppressWarnings("deprecation")
public class CSVResourceService implements ResourceService {

    @Autowired
    private WordRepository wordRepository;

    @Autowired
    private WordRepositoryHelper wordRepositoryHelper;

    private static final String HEADER_ID = "id";
    private static final String HEADER_VALUE = "value";
    private static final String HEADER_CATEGORY = "category";

    @Override
    public Boolean save(MultipartFile file, Boolean replace) throws Exception {

        // some other code

        if (replace) {
            wordRepositoryHelper.truncateWordTable();
        }

        //some other code
    }
}

Any guidance on the issue or suggestion.

Upvotes: 7

Views: 22039

Answers (1)

user5412293
user5412293

Reputation:

Thanks a lot for all the suggestions in the comments. As you might have guessed this is the first time I am dealing with memory issues and since it was happening in production I panicked a bit.

So the real problem was not really JPA. It was more like a combination of problems and issues. And like some of the comments suggested I had quite a few candidates to blame for my memory problems:

  • JPA
  • Tika
  • Opencv
  • Tesseract

This is how I solved my issue:

  1. Educate. Go out there and learn a bit about the problem and how it could be solved. Here are a few links I used:
    https://www.toptal.com/java/hunting-memory-leaks-in-java
    https://developers.redhat.com/blog/2014/08/14/find-fix-memory-leaks-java-application/
    https://app.pluralsight.com/player?course=java-understanding-solving-memory-problems
  2. Use the new knowledge and pull some graphs and understand your memory situation. Trust me this exploration revealed a whole bunch of stuff that I did not know was happening. (like some Chinese characters array, my guess Tesseract needs it for OCR)
    enter image description here
    enter image description here
  3. Then use -Xms256m -Xmx512m when executing your jar so you can narrow and see who to blame. Also it will limit the resources in the server.
  4. All this lead me to two possible causes of leak and trouble in general.
    1) Using POI or Tika to extract text, stream-to-stream without loading the entire file in memory
    2) Memory Leak from iterating Opencv frames

And that's it I think I am ok now.

Thanks to all for the help.

Upvotes: 12

Related Questions