CODEWITHSUNDEEP
amazon-s3botoduplicityduplicity-backup
pallago
pallago

Reputation: 431

Duplicity SSLEOFError: EOF occurred in violation of protocol

I have duplicity running for years, stable and without problems. Now I encountered the following problem when I backup my files to Amazon S3.

All files in the format "duplicity-full.YYYYMMDD1ThhmmssZ.volX.difftar.gpg" are written to Amazon S3 successfully. At least I can see them in the directory/bucket.

Unfortunately, when uploading the "duplicity-full-signatures.YYYYMMDD1ThhmmssZ.sigtar.gpg" an error occurs stating:

Backtrace of previous error: Traceback (innermost last):
  File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 387, in inner_retry
    return fn(self, *args)
  File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 574, in move
    self.__do_put(source_path, remote_filename)
  File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 544, in __do_put
    self.backend._put(source_path, remote_filename)
  File "/usr/lib/python3/dist-packages/duplicity/backends/_boto_single.py", line 265, in _put
    self.upload(source_path.name, key, headers)
  File "/usr/lib/python3/dist-packages/duplicity/backends/_boto_single.py", line 319, in upload
    key.set_contents_from_filename(filename, headers,
  File "/usr/local/lib/python3.10/dist-packages/boto/s3/key.py", line 1375, in set_contents_from_filename
    return self.set_contents_from_file(fp, headers, replace, cb,
  File "/usr/local/lib/python3.10/dist-packages/boto/s3/key.py", line 1307, in set_contents_from_file
    self.send_file(fp, headers=headers, cb=cb, num_cb=num_cb,
  File "/usr/local/lib/python3.10/dist-packages/boto/s3/key.py", line 760, in send_file
    self._send_file_internal(fp, headers=headers, cb=cb, num_cb=num_cb,
  File "/usr/local/lib/python3.10/dist-packages/boto/s3/key.py", line 957, in _send_file_internal
    resp = self.bucket.connection.make_request(
  File "/usr/local/lib/python3.10/dist-packages/boto/s3/connection.py", line 667, in make_request
    return super(S3Connection, self).make_request(
  File "/usr/local/lib/python3.10/dist-packages/boto/connection.py", line 1070, in make_request
    return self._mexe(http_request, sender, override_num_retries,
  File "/usr/local/lib/python3.10/dist-packages/boto/connection.py", line 1030, in _mexe
    raise ex
  File "/usr/local/lib/python3.10/dist-packages/boto/connection.py", line 939, in _mexe
    response = sender(connection, request.method, request.path,
  File "/usr/local/lib/python3.10/dist-packages/boto/s3/key.py", line 856, in sender
    http_conn.send(chunk)
  File "/usr/lib/python3.10/http/client.py", line 999, in send
    self.sock.sendall(data)
  File "/usr/lib/python3.10/ssl.py", line 1266, in sendall
    v = self.send(byte_view[count:])
  File "/usr/lib/python3.10/ssl.py", line 1235, in send
    return self._sslobj.write(data)
 ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:2426)

Attempt of move Nr. 1 failed. SSLEOFError: EOF occurred in violation of protocol (_ssl.c:2426)

After 5 attemps, the upload is stopped. Checking the upload online monitor of my router I see that a file is uploaded, but only for short time (like 10 sec). In addition, I cannot find the "duplicity-full-signatures.YYYYMMDD1ThhmmssZ.sigtar.gpg" in the S3 directory/bucket. This problem only occurs with one big directory (~1 TB). Other directories work without any problems (same user, same password, same keys etc.)

I have Ubuntu 22.04.4 LTS with kernel 5.15.0-116-generic with duplicity 0.8.21.

Any ideas? How to debug? And how to find the problem? I got the error message running

duplicity --verbosity d

Upvotes: 0

Views: 152

Answers (3)

pallago
pallago

Reputation: 431

So, here is the answer for somebody having the same or a similar problem:

  1. Install boto3: python3 -m pip install boto3

  2. Change the TARGET (I found here: duplicity error only during retrieve: boto.exception.NoAuthHandlerFound and here: https://serverfault.com/questions/704442/duplicity-backup-to-s3-backendexception), i.e. remove the hostname:

    FROM: s3://HOSTNAME/bucket/subfolder

    TO: boto3+s3://bucket/subfolder

Upvotes: 1

pallago
pallago

Reputation: 431

When trying to run with boto3+s3:// I get:

Temp has 41315815424 available, backup will use approx 272629760.
Backtrace of previous error: Traceback (innermost last):
  File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 387, in inner_retry
    return fn(self, *args)
  File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 607, in list
    return [tobytes(x) for x in self.backend._list()]
  File "/usr/lib/python3/dist-packages/duplicity/backends/s3_boto3_backend.py", line 167, in _list
    self.reset_connection()
  File "/usr/lib/python3/dist-packages/duplicity/backends/s3_boto3_backend.py", line 95, in reset_connection
    self.s3.meta.client.head_bucket(Bucket=self.bucket_name)
  File "/usr/local/lib/python3.10/dist-packages/botocore/client.py", line 565, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.10/dist-packages/botocore/client.py", line 1017, in _make_api_call
    raise error_class(parsed_response, operation_name)
 botocore.exceptions.ClientError: An error occurred (403) when calling the HeadBucket operation: Forbidden

Attempt of list Nr. 1 failed. ClientError: An error occurred (403) when calling the HeadBucket operation: Forbidden
Backtrace of previous error: Traceback (innermost last):
  File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 387, in inner_retry
    return fn(self, *args)
  File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 607, in list
    return [tobytes(x) for x in self.backend._list()]
  File "/usr/lib/python3/dist-packages/duplicity/backends/s3_boto3_backend.py", line 170, in _list
    for obj in self.bucket.objects.filter(Prefix=self.key_prefix):
 AttributeError: 'NoneType' object has no attribute 'objects'

Attempt of list Nr. 2 failed. AttributeError: 'NoneType' object has no attribute 'objects'

Upvotes: 0

ede-duply.net
ede-duply.net

Reputation: 543

sounds like you hit the upper file size limit of Amazon S3 for single put operations (5GB). unfortunately the manifest is not split to volume sizes even in recent duplicity.

but, as Amazon S3 documentation states files (objects) up to 5TB can be uploaded with multipart upload API, which is the default in current duplicity 3.x as it uses the current boto3 python module.

your old duplicity already had this backend integrated, albeit only as alternative only. you can use it using the boto3+s3:// target url scheme. you will need to have package python3-boto3 installed.

Upvotes: 0

Related Questions