noempty should upload existing files to bucket?

[karan]

Describe the issue
I expected that mounting gcsfuse on a directory with existing files would result in those files being synced to GCS. However, the files are not present in the dir when gcsfuse is started (my bucket is empty).

To Collect more Debug logs
Steps to reproduce the behavior:

1. Please make sure you have no other security, monitoring, background processes which can offend the FUSE process running. Possibly reproduce under a fresh/clean installation.
2. Please rerun with --debug_fuse --debug_fs --debug_gcs --debug_http --foreground as additional flags to enable debug logs.
3. Monitor the logs and please capture screenshots or copy the relevant logs to a file (can use --log-format and --log-file as well).
4. Attach the screenshot or the logs file to the bug report here.
5. If you're using gcsfuse with any other library/tool/process please list out the steps you took to reproduce the issue.

~/mount-folder$ echo "hello" > foo.txt

$ gcsfuse -o rw,nonempty --debug_fuse --debug_fs --debug_gcs --debug_http --foreground --implicit-dirs $BUCKET ~/mount-folder/

A few seconds later, the file is "deleted" (ls doesn't show it if gcsfuse is running, but it does if gcsfuse is stopped).

Logs:

{"name":"root","levelname":"INFO","severity":"INFO","message":"Start gcsfuse/1.0.1 (Go version go1.20.5) for app \"\" using mount point: /home/user/mount-folder/\n","timestampSeconds":1692055966,"timestampNanos":540149851}
{"name":"root","levelname":"INFO","severity":"INFO","message":"Opening GCS connection...\n","timestampSeconds":1692055966,"timestampNanos":540225421}
{"name":"root","levelname":"INFO","severity":"INFO","message":"Creating a mount at \"/home/user/mount-folder/\"\n","timestampSeconds":1692055966,"timestampNanos":543620609}
{"name":"root","levelname":"INFO","severity":"INFO","message":"Creating a new server...\n","timestampSeconds":1692055966,"timestampNanos":543676400}
{"name":"root","levelname":"INFO","severity":"INFO","message":"Set up root directory for bucket $BUCKET\n","timestampSeconds":1692055966,"timestampNanos":543692277}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"gcs: Req              0x0: \u003c- ListObjects(\"\")\n","timestampSeconds":1692055966,"timestampNanos":543706580}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"gcs: Req              0x0: -\u003e ListObjects(\"\") (164.540755ms): OK\n","timestampSeconds":1692055966,"timestampNanos":708261265}
{"name":"root","levelname":"INFO","severity":"INFO","message":"Mounting file system \"$BUCKET\"...\n","timestampSeconds":1692055966,"timestampNanos":708410485}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Beginning the mounting kickoff process\n","timestampSeconds":1692055966,"timestampNanos":708494846}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Parsing fuse file descriptor\n","timestampSeconds":1692055966,"timestampNanos":708536589}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Preparing for direct mounting\n","timestampSeconds":1692055966,"timestampNanos":708543068}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Successfully opened the /dev/fuse in blocking mode\n","timestampSeconds":1692055966,"timestampNanos":708572955}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Starting the unix mounting\n","timestampSeconds":1692055966,"timestampNanos":708609568}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Directmount failed. Trying fallback.\n","timestampSeconds":1692055966,"timestampNanos":708627188}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Creating a socket pair\n","timestampSeconds":1692055966,"timestampNanos":708695464}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Creating files to wrap the sockets\n","timestampSeconds":1692055966,"timestampNanos":708727252}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Starting fusermount/os mount\n","timestampSeconds":1692055966,"timestampNanos":708742879}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Wrapping socket pair in a connection\n","timestampSeconds":1692055966,"timestampNanos":710532113}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Checking that we have a unix domain socket\n","timestampSeconds":1692055966,"timestampNanos":710592272}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Read a message from socket\n","timestampSeconds":1692055966,"timestampNanos":710604488}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Successfully read the socket message.\n","timestampSeconds":1692055966,"timestampNanos":710618163}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Converting FD into os.File\n","timestampSeconds":1692055966,"timestampNanos":710628138}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Completed the mounting kickoff process\n","timestampSeconds":1692055966,"timestampNanos":710660210}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Creating a connection object\n","timestampSeconds":1692055966,"timestampNanos":710673050}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000002        connection.go:416] \u003c- init\n","timestampSeconds":1692055966,"timestampNanos":710996856}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000002        connection.go:498] -\u003e OK ()\n","timestampSeconds":1692055966,"timestampNanos":711032812}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Successfully created the connection\n","timestampSeconds":1692055966,"timestampNanos":711051389}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Waiting for mounting process to complete\n","timestampSeconds":1692055966,"timestampNanos":711066841}
{"name":"root","levelname":"INFO","severity":"INFO","message":"File system has been successfully mounted.\n","timestampSeconds":1692055966,"timestampNanos":711083128}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000004        connection.go:416] \u003c- GetInodeAttributes (inode 1, PID 1762)\n","timestampSeconds":1692055973,"timestampNanos":500172064}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000004        connection.go:498] -\u003e OK ()\n","timestampSeconds":1692055973,"timestampNanos":500480409}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000006        connection.go:416] \u003c- GetXattr (inode 1, name \"security.selinux\", PID 27132, name security.selinux)\n","timestampSeconds":1692055973,"timestampNanos":782008471}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000006        connection.go:500] -\u003e Error: \"function not implemented\"\n","timestampSeconds":1692055973,"timestampNanos":782166805}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000008        connection.go:416] \u003c- OpenDir (inode 1, PID 27132)\n","timestampSeconds":1692055973,"timestampNanos":782590176}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000008        connection.go:498] -\u003e OK ()\n","timestampSeconds":1692055973,"timestampNanos":782714333}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x0000000a        connection.go:416] \u003c- ReadDir (inode 1, PID 27132)\n","timestampSeconds":1692055973,"timestampNanos":782837321}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"gcs: Req              0x1: \u003c- ListObjects(\"\")\n","timestampSeconds":1692055973,"timestampNanos":782914515}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"gcs: Req              0x1: -\u003e ListObjects(\"\") (34.895007ms): OK\n","timestampSeconds":1692055973,"timestampNanos":817814151}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x0000000a        connection.go:498] -\u003e OK ()\n","timestampSeconds":1692055973,"timestampNanos":817888715}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x0000000c        connection.go:416] \u003c- ReleaseDirHandle (PID 0)\n","timestampSeconds":1692055973,"timestampNanos":818017487}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x0000000c        connection.go:498] -\u003e OK ()\n","timestampSeconds":1692055973,"timestampNanos":818134495}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x0000000e        connection.go:416] \u003c- OpenDir (inode 1, PID 27135)\n","timestampSeconds":1692055974,"timestampNanos":822768898}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x0000000e        connection.go:498] -\u003e OK ()\n","timestampSeconds":1692055974,"timestampNanos":823035572}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000010        connection.go:416] \u003c- GetInodeAttributes (inode 1, PID 27135)\n","timestampSeconds":1692055974,"timestampNanos":823130135}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000010        connection.go:498] -\u003e OK ()\n","timestampSeconds":1692055974,"timestampNanos":823274480}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000012        connection.go:416] \u003c- ReadDir (inode 1, PID 27135)\n","timestampSeconds":1692055974,"timestampNanos":823364910}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"gcs: Req              0x2: \u003c- ListObjects(\"\")\n","timestampSeconds":1692055974,"timestampNanos":823424255}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"gcs: Req              0x2: -\u003e ListObjects(\"\") (43.3444ms): OK\n","timestampSeconds":1692055974,"timestampNanos":866772673}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000012        connection.go:498] -\u003e OK ()\n","timestampSeconds":1692055974,"timestampNanos":866882564}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000014        connection.go:416] \u003c- ReleaseDirHandle (PID 0)\n","timestampSeconds":1692055974,"timestampNanos":866983891}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000014        connection.go:498] -\u003e OK ()\n","timestampSeconds":1692055974,"timestampNanos":867057195}

System (please complete the following information):

• OS: 22.04.1-Ubuntu
• Platform: GCE VM
• Version: gcsfuse version 1.0.1

Additional context
Add any other context about the problem here.

SLO:
24 hrs to respond and 7 days to close the issue.

[Tulsishah]

Hi @karan ,

Thanks for reaching out to us.

By default, fuse does not allow mounting a bucket in a non-empty directory. This is because it can lead to data corruption if the files in the directory already exist and have the same names as the files in the bucket.

However, you can use the -o nonempty option to force fuse to mount the bucket in a non-empty directory. This option tells fuse to overwrite any existing files in the directory with the corresponding files from the bucket.

I think this is default behaviour of fuse only and not related to gcsfuse. Still I'll get back to you if we'll consider it as feature request.

Thanks,
Tulsi Shah.