CODEWITHSUNDEEP
go
Leonardo Esquivel
Leonardo Esquivel

Reputation: 67

Get file path in Golang

I need to read all files inside a directory, the structure is something like:

/data
   /folder1
     somefiles.txt
   /folder2
     /folder2.1
       somefiles.txt
   /folder3
       somefiles.txt
    ...

I tried the walk method, but walk reads the file and when it's done with the file it deletes the information and moves on to the next one. I really need to save it in an object (I'm using a slice of map) with all the data because I need to use an api endpoint and it's faster when I insert a lot of data than one by one (Inside the directories there are thousands of files so one by one takes a lot of time)

My idea is do a recursive function(like walk) but I don't know how to know the file path or I don't know if I can access with pointers inside walk function (I'm really new with GO) and save the file data in a global variable or something like that

Upvotes: 0

Views: 2999

Answers (1)

Nicholas Carey
Nicholas Carey

Reputation: 74177

To use fs.WalkDir(), you just need to have your handler close over the variable(s) that you need to have populated when the execution of fs.WalkDir() is complete.

This, for instance, recursively walks the specified directory tree. Directories are ignored. A map[string][]byte is populated, keyed by the fully-qualified path of each file, containing the contents of each file.

package main

import (
  "fmt"
  "io/fs"
  "os"
  "path"
  "path/filepath"
  // "path"
)

func main() {
  root := "/path/to/root/directory"

  files := map[string][]byte{}

  handler := func(p string, d fs.DirEntry, err error) error {

    switch {
    case err != nil:
      return err
    case shouldSkipDir(p, d):
      return fs.SkipDir
    case d.IsDir():
      return nil
    case shouldSkipFile(p, d):
      return nil
    default:
      if fqn, e := filepath.Abs(path.Join(root, p)); e != nil {
        return e
      }
      buf, e := os.ReadFile(fqn)
      if e != nil {
        return e
      }
      files[fqn] = buf
    }

    return nil
  }

  err := fs.WalkDir(os.DirFS(root), ".", handler)
  if err != nil {
    panic(err)
  }

  for fqn, contents := range files {
    fmt.Printf("%s: %d bytes", fqn, len(contents))
  }

}

func shouldSkipDir(path string, d fs.DirEntry) (skip bool) {
  // TODO: set skip to true to indicate that this directory should be skipped.
  // If the current direcotry entry is a file and this returns true, then
  // the remainder of its containing directory are skipped.
  return skip
}

func shouldSkipFile(path string, d fs.DirEntry) (skip bool) {
  // TODO: set skip to true to ignore this file
  return skip
}

Upvotes: 1

Related Questions