Issues with uploading manifest.xml in Python

Nikolas N

Hello,

I am having trouble with the Panopto Video Upload API, trying to upload a single video using a manifest.xml file.

I have looked over the example code here: https://github.com/Panopto/upload-python-sample/blob/master/simplest/panopto_uploader.py#L112 And I have my own simplified implementation based on this.

I am confident my manifest is well-formatted and I have tried many different changes in the textual format here. it seems my upload process, no matter what I do, results in this error on Panopto:

2025-02-13 12:51:01 PM: Error processing upload: : (034a4817-34c8-4756-935a-3dc6c209e367.xml: Data at the root level is invalid. Line 1, position 1.)
2025-02-13 12:51:00 PM: Error processing upload: : (034a4817-34c8-4756-935a-3dc6c209e367.xml: Data at the root level is invalid. Line 1, position 1.)
2025-02-13 12:50:57 PM: Error processing upload: : (034a4817-34c8-4756-935a-3dc6c209e367.xml: Data at the root level is invalid. Line 1, position 1.)
2025-02-13 12:50:56 PM: EventTarget 10a3950c-6718-41cd-ae78-b28301262503 created.
2025-02-13 12:50:56 PM: Session 6528dec9-dc08-4285-a20e-b283012624e6 created.

Panopto Support has told me that my manifest file starts with the characters "1d1" rather than the expected "<" character. So I've been trying different encodings and options, but everything I do results in this same parse error on Panopto at line 1, position 1.

Here is some code I have tried using AWS multipart upload to upload this manifest:

        buff = io.BytesIO(manifest)

        mpu = self.s3.create_multipart_upload(
            Bucket=self.target.bucket_name, Key=key_name,
            ContentType='text/xml')
        mpu_id = mpu['UploadId']
        part = self.s3.upload_part(
            Body=buff, Bucket=self.target.bucket_name,
            Key=key_name, UploadId=mpu_id, PartNumber=1)
        parts = [{'PartNumber': 1, 'ETag': part['ETag']}]
        result = self.s3.complete_multipart_upload(
            Bucket=self.target.bucket_name,
            Key=key_name, UploadId=mpu_id, MultipartUpload={'Parts': parts})
        print('result', result)

And my usual implementation of this which worked for years but randomly started failing last month uses S3's upload_fileobj() method: https://github.com/ccnmtl/pypanopto/blob/main/panopto/upload.py#L190

It's hard to debug this without seeing the resulting XML file on Panopto's S3 bucket - is there any way I can access this XML file once it's uploaded to help me debug this?

Nikolas N

So we have tracked down the cause of this issue - it wasn't our code but rather a library update from AWS causing this.

Last month we updated these Python dependencies:

s3transfer==0.11.2
botocore>=1.12.0,<1.37.0
boto3>=1.12.0,<1.37.0

Downgrading these libraries to their previous versions seems to have resolved the issue:

s3transfer==0.10.4
botocore>=1.12.0,<1.36.0
boto3>=1.12.0,<1.36.0

Now we are looking through the changelogs for boto3 and botocore to see why this caused uploads to stop working on Panopto's S3 bucket.