Upload doesn't seem to be using UCS xml file

Thom Brooks · January 20

Given a session / upload id, I'd appreciate it if someone could look at the logs on your end to see if there's any obvious reason why this doesn't seem to be working - I think it looks almost exactly like the xml files included with the sample code.

Our hosted instance is chicagotrading.hosted.panopto.com

My most recent session was b83253c5-9f7d-421d-9d67-b3d8014a6b14

Here is the content of one of our ucs.xml files (is there a code formatting tag on this forum?):

<?xml version="1.0" encoding="utf-8"?>

<Title>Profit Sharing and 401k Match 01-28-2019</Title>

<Video>

<Type>Primary</Type>

</Video>

<Video>

<Type>Secondary</Type>

</Video>

</Videos>

</Session>

(note: we have tried a version without the empty transcripts / cuts / tableofcontents tags; we weren't sure if these were required.)

This is where we're trying to put the first batch of test uploads:
https://chicagotrading.hosted.panopto.com/Panopto/Pages/Sessions/List.aspx#folderID=%22f9cd6bc5-0a3c-44de-9304-b3d80013a143%22

We see empty files show up in the desired folder - they don't use the title in the ucs.xml file, they're just labeled with today's date and time UTC, and they say 'no content' - if you click 'edit' in the Web UI, they want you to add the first piece of content.

Our main questions at this point are:

Does account 'svc-recorder' have UCS upload capability enabled?
Are there additional permissions or configuration required for UCS manifest processing? (it is an admin and a creator, and it has creator permissions for the parent folder, which should be inherited)
Can you review server-side logs for upload 7e8c8531-09d2-4c37-bf1a-b3d8014695e7 to see why manifest was ignored?

Thank you!

Thom Brooks · January 21

Further tests:

Tried a different admin user with their own API key; no change.
Tried uploading into a different folder closer to the root level; no change.

Adis Hrnjica · January 21

Hi Thom,

What are you using to perform the upload?
I can see the following message in the logs:

Message: User does not have permission to perform the requested action. UploadId: 7e8c8531-09d2-4c37-bf1a-b3d8014695e7 SessionId: Reason: User not found

Thanks,

Adis

Thom Brooks · January 26

Adis,

We are using Powershell, but it is based off of the UCS upload sample code in the GitHub archive.

I thought I had figured out our issue: when we set up our site, our rep had us create accounts for ourselves as Okta\User Name but also a 'break glass' (local Panopto) with the same User name. When I look at the list of API Clients, it doesn't put Okta\User Name - you have to click on the user in order to see that detail. So I had created an API client from my Okta user but provided the user/pass of the local user.

I fixed that today: I made an API Client for the local user, (which is both a site admin and a creator on the folder where I'm trying to upload via API), but it still has the same symptoms.

I'm going to attach a log file from the session, the Mediasite xml file we started with, the manifest file we generated, and ask you to look again at the logs. No matter what we try and tweak, it always seems to do the same thing - lets us upload the mp4 files and our manifest, but doesn't use any info from the manifest. So we end up with an empty presentation with the wrong title (today's date and time UTC), no content, and wrong create date/time.

Among other things we have tried:

With and without empty attributes (Transcripts, Cuts, Images, etc) included - the xsd says these are optional but the samples show them;
Uploading as different users/API keys
Uploading to different folders
With and without CRC32 checksums
etc

Thank you - TB

550d359583e549148fe5d18f3bb868bd1d.zip

Adis Hrnjica · January 27

Hi Thom,

I can see this in the session log:
ucs.xml: Data at the root level is invalid. Line 1, position 1.

This means the XML parser couldn’t read the very first byte of ucs.xml. In practice, that almost always comes from bad file encoding (PowerShell defaulting to UTF‑16 while the XML header says UTF‑8), or non‑XML bytes at the start of the file.

PowerShell Out-File / Set-Content default to UTF‑16. If your file starts with a UTF‑16 BOM or null bytes while the header says encoding="utf-8", Panopto’s XML parser throws exactly this error.

How to confirm this:

Run these on the machine that generates the file:
# Show first 4 bytesGet-Content "I:\CTCUBackup\...\ucs.xml" -Encoding Byte -TotalCount 4

UTF‑8 (with BOM) starts with: EF BB BF
UTF‑16 LE starts with: FF FE
If you see FF FE, that’s the problem.

How to fix it:

Write the XML as UTF‑8 without BOM and make sure the file begins immediately with <?xml ...?>:
Example patterns that work:

Set-Content -Encoding utf8 (PowerShell 7+ writes UTF‑8 without BOM)
In Windows PowerShell you could potentially use:
[System.IO.File]::WriteAllText($path, $xmlString, (New-Object System.Text.UTF8Encoding($false)))

Additionally also double check:

No blank lines or other characters before the XML declaration.
No hidden nulls or stray characters.

Once the encoding is fixed, the manifest should parse and the session should pick up title/date/streams.

I hope this helps.

Thanks,

Adis

Thom Brooks · January 27

Adis,

By any chance did you happen to look at the files I attached to my last post? I ask because I included my ucs.xml file, and it WAS UTF-8. Granted, it had the BOM (which as you can see from this screenshot I am now correcting - I have to use PS5.1 for this, and we have run into the 8/16 thing before) but it appears that even AFTER fixing this issue, the file is STILL being rejected, perhaps for another reason now?

I have attached a NEW set of these files; I note that they use CR/LF linefeeds. (Also, the moon is in the waxing gibbous phase, if that has any impact.) Could you please kindly open them and tell me if you see anything else wrong with them?

May I respectfully suggest (if for no other reason than so we don't have to waste as much of your time):

Adding a mention of the UTF-8 with NO BOM to the documentation and the UCS spec pages
Making access to the logs available so we can self-troubleshoot
Detecting common issues like this (sounds like we are not the first to run into this) and either throwing an error, or quietly fixing the issue for the customer
Providing a validation tool

Thank you - TB

550d359583e549148fe5d18f3bb868bd1d.zip

Adis Hrnjica · January 27

Hi Thom,

Do you have the ability to send me your powershell script? Strip all the info that are sensitive and that I don't need so I can troubleshoot this on my machine. Unfortunately we are unable to share the access to the logs with you, but I can determine what's the issue with the request that is coming from powershell.

I'll work with the team to mention the UTF8 stuff, and see if we could offer additional guidance.

Thanks,

Adis

Thom Brooks · January 27

Adis,

No worries - we keep the credentials on a Secret server, so the only thing you would see is a reference to a specific secret ID.

However, is there a place where we can do these kinds of file uploads? Not sure what rules our company has about posting code to a publicly accessible forum.

Thank you - TB

Adis Hrnjica · January 27

Hi Thom,

Yes, lets switch to our support. Create a support ticket for this or send email to [email protected]
So we can work on this, and you'll have a secure place to upload the needed files.

Thanks,
Adis

Welcome to the Panopto Community

Upload doesn't seem to be using UCS xml file

Answers