A few scripts that can download and upload files to SharePoint/OneDrive/Teams.

• Overview
• Required Script Setup
• Setup Microsoft Azure AD App
• Finding Your Drive ID
• Finding Your Folder Path
• Acquiring an MFA Secret
• Python Package Installation
• Firefox & Geckodriver Setup
• Common Questions & Issues

These Python scripts allow you to download or upload files to a SharePoint/OneDrive/Teams drive that an account has access to. To achieve this, the script primarily uses Microsoft Graph API, the Microsoft Authentication Library (msal), Firefox (the web browser), and Selenium along with a few other Python libraries. The script is ran simply by running python3 sharepoint_downloader.py for the download, and python3 sharepoint_uploader.py for the upload.

Note: Any time I mention the sharepoint_downloader.py script in the tutorial, the same also applies to sharepoint_uploader.py

This script optionally offers numerous flags/arguments you can add to the end of the script to change how it will run. To view the general help page for these arguments, add the flag -h or --help to the end of the script (like so: python3 sharepoint_downloader.py -h or --help). For more information on these runtime arguments, expand the section below.

This script requires a decent amount of pre-configuration before it will work, with this File Handling in SharePoint with Python tutorial being the main inspiration for this script. However, I had a few issues following this tutorial (no information on drive_id's and token generation didn't work with MFA), so a full setup tutorial for this script is included below.

Note: Some of the images may look a bit compressed due to resizing them to fit the narrow GitHub README column, so if you have any issues seeing anything, you can click on the image to enlarge them.

Lastly, if you want to get the six-digit MFA code that you need during setup without using an app on your phone, the mfa_code_generator.py file can generate your six-digit code and tell you how many more seconds it is valid for. All you need is the MFA_SECRET variable filled out in your msal_config.env file.

Most of the setup for this script will eventually be entered into a msal_config.env file with the following variables as seen below and also in sample.env:

CLIENT_ID =         "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"
AUTHORITY_URL =     "https://login.microsoftonline.com/cccccccc-cccc-cccc-cccc-cccccccccccc"
M365_DRIVE_ID =     "##################Ml0MFG46i5yVcjFFrjNUDOU7RZgloc7UUnFE############"
M365_FOLDER_PATH =  "Network Operations/On-Call & Scheduled Work"
M365_FILENAME =     "NetOps Work Tracker.xlsx"
MFA_SECRET =        "2w2h5q6ztwbb75rx"
M365_USERNAME =     "[email protected]"
M365_PASSWORD =     "hunter2"

Without all these variables filled out, the script will not work (unless you use certain runtime arguments). You will need to create the msal_config.env file as one of the first things during setup. You can either do this manually or run sharepoint_downloader.py without a msal_config.env file in your script's directory, and the script will detect that you are missing the file and ask if you want to make a blank msal_config.env file.

If any of your variables in msal_config.env are not filled out or are very obviously wrong, the script will report which variables are blank/wrong, and then exit the script. I will explicitly mention the variables in the tutorial when they appear, so just remember that they will need to be eventually filled out and that the script will not work unless all eight variables are filled out.

First, an app in Microsoft Azure AD is required. This app is what will be giving this script permission to view and download files from your Microsoft 365 account. To start this process, head over to the Azure Portal and login into your account. After logging in, click on the Azure Active Directory button.

After entering Azure Active Directory, select App registrations and then New registration to begin the app creation process.

Next, you must give the app a name and select Accounts in this organizational directory only ({Tenant Name} only - Single tenant) as you are downloading files from the same organizational directory this app is in. This essentially means you will only be able to download files from within your organization. I think you may be able to select the multitenant option, but this will open your API permissions to ANY Microsoft 365 account, look at the Common Questions & Issues for a more detailed discussion of single vs multitenant apps.

Finally, at the bottom of the page is the most important step in this guide. For the Redirect URI, select Public client/native (mobile & desktop) from the drop-down list and type in http://localhost:9999/tokenCallback. The script WILL NOT WORK if the redirect URI platform configuration is set to anything but Public client/native (mobile & desktop). If you already have an application on port 9999, look at the Common Questions & Issues for a fix.

The reason behind these selections is that mobile & desktop are the only platform type that allows custom redirect URIs, and the redirect points to a non-existent localhost server in order to substring the end of the URL which will have authentication variables. Look at loginProcess() and createAuthResponseDict() if you want to know more about the specifics.

After registering your app, you will be brought to the app overview page which will have two credentials that are needed in msal_config.env, the Application/Client ID and the Tenant ID. Copy the Application/Client ID to the CLIENT_ID variable and copy the Tenant ID to the end of https://login.microsoftonline.com/{tenant_id} for the AUTHORITY_URL variable.

Next, you need to add the API permissions needed to download files from a SharePoint/OneDrive/Teams site. These permissions are called "scopes" and I will be referring to them as such henceforth. You may notice that the User.Read scope is already added to your new app. You can remove it if you want, but it doesn't hurt to have it either. In order to start adding a new scope, first select the API permissions menu selection and then click the Add a permission button.

Once there, select the Microsoft Graph button. As you can see, there are other APIs one could use for other scripts, but for this script, we will be using Microsoft Graph.

Next, select the Delegated permissions button as we will be signing into a user's account for this script.

Afterwards, search for sites and add the Sites.Read.All scope to the app.

Do the same again and search for files and add the Files.ReadWrite.All scope to the app.

If you have done this correctly you should have something that looks like the image below. Make sure you have both Sites.Read.All and Files.ReadWrite.All as configured scopes. Sites are the top-level objects in SharePoint/OneDrive/Teams drives, and the file scope allows you to download and upload files within those sites.

Next, you need to Expose an API. This is what specifies the scopes within the access token that is used for authentication. Before the API performs a call, it checks to see what scopes are within the token, and if the scopes are missing, the API will refuse to perform the requested operation. In order to add these scopes, first select the Expose an API menu selection and click the Add a Scope button.

You will then be asked to set an Application ID URI. It should autocomplete with your Client ID, so after confirming that it is correct, select Save.

Next, you will need to type in the scope name exactly the same as it is displayed. Keep all capitals the same and don't add any spaces before, in, or after the scope name. Specifically, we will be adding the Sites.Read.All and the Files.ReadWrite.All scopes we just defined.

First enter in Sites.Read.All for the Scope name. After entering the scope name select Admins and users under the Who can consent section. Next, add a consent display name and description for both the admin and the user. These can really be anything you want, but it would be useful to give a brief name and description for each scope. Finally, make sure you select the Enabled state at the bottom and finally click Add Scope. Repeat all of these steps for the Files.ReadWrite.All scope as well.

After adding these scopes, you should see them both in the Scopes defined by this API section, as seen below.

Finally, select the Add a client application button under the Authorized client applications section. Click the checkboxes for the two Sites.Read.All and Files.ReadWrite.All scopes you just exposed and select Add application.

If you have done everything correctly so far, your Expose an API screen should look similar to the screenshot below, and you should have your CLIENT_ID and AUTHORITY_URL variables entered in your msal_config.env file.

Finding your drive_id is what made me initially start writing this script, as the tutorial linked at the start mentions drive_id's, but never tells you how to find them. Additionally, the only Microsoft response I could find on some Power Automate forum has this super roundabout way of getting your drive_id, so I managed to find two solutions, one using Microsoft's Graph Explorer and one using the sharepoint_downloader.py python script.

Make sure to do two things before you start attempting to find your drive_id.

• Firstly, make sure you have recently opened whatever file you want to download in SharePoint/OneDrive/Teams.
• Secondly, open up the file you wish to download and share it with the account that will be used in this script.

Both the script method and the Microsoft Graph Explorer method can look at recently opened files to find your drive_id or can look through all files that are shared with your account to find your drive_id. The python script method will attempt both of these automatically, while the Microsoft Graph Explorer method will require a few extra clicks to perform both.

The first method here is the Graph Explorer method, which is a bit more user friendly compared to the script method. To start, you first need to browse to Microsoft's Graph Explorer website and sign in to your account in the top right. This website lets you run sample Microsoft Graph API calls on your actual account as both a development and testing tool.

After logging in, first click on Resources on the top left side of the page. Next, scroll down to the drive (not drives, they are different) resource and select the microsoft.graph.recent() API query. Finally, click Run query in the top right.

After running the query, you should get a giant JSON response at the bottom of the page.

First, you will be looking for the value of one of the name keys to match the filename you are looking for. After finding that key-value pair, scroll down until you see the remoteItem key. Within all those values should be another key-value pair for parentReference. Lastly, within the parentReference values should be a key driveId, which will have the value of your drive_id that you are looking for.

Overall, your JSON query would look something like this ["value"][i]["remoteItem"]["parentReference"]["driveId"] with the i representing each item you've recently viewed. Copy this driveId value to the M365_DRIVE_ID variable in your msal_config.env file.

If that didn't work, go back to Sample Queries, search for shared in the search bar, select the OneDrive files shared with me resource, and click Run query.

Then just like before, you get a giant JSON response at the bottom of the page. Follow the same ["value"][i]["remoteItem"]["parentReference"]["driveId"] path that you followed before to find your M365_DRIVE_ID variable. Below is an example of finding the driveId variable for a shared word doc named "Dummy Test File.docx".

If for some reason you are unable to access the Graph Explorer app in your organization, but do still have the Sites.Read.All and Files.ReadWrite.All scopes, you can instead perform the two API calls through the script. This second solution using the script requires you to jump ahead a bit and download all the python packages, which is outlined in the Python Package Installation step here. Additionally, if you will be using MFA, you will also need to follow the steps for Acquiring an MFA Secret here. After downloading all the python packages and setting up MFA if needed, make sure CLIENT_ID, AUTHORITY_URL, MFA_SECRET (if used), M365_USERNAME, and M365_PASSWORD are filled out.

Next, run the sharepoint_downloader.py script with either the -D or --driveid flags (like so: python3 sharepoint_downloader.py -D or python3 sharepoint_downloader.py --driveid). The script will take about a minute to generate a token and then perform the exact same two Microsoft Graph API calls I outlined above, one for recently viewed files and one for files shared with you.

If your JSON output looks similar to one of the two images above, then all you need to do is follow the ["value"][i]["remoteItem"]["parentReference"]["driveId"] path to find your M365_DRIVE_ID variable. I would highly recommend copy-pasting the output into some sort of online JSON formatter for easier viewing.

After completing this step, you should have three variables filled out in your msal_config.env file; CLIENT_ID, AUTHORITY_URL, and M365_DRIVE_ID. If you ended up needing to use the script method to generate your drive_id, then you can skip the steps below you already had to follow.

In this section you will be setting your M365_FOLDER_PATH and M365_FILENAME variables. This section is rather simple but can be easy to mess up.

First, navigate to the file you wish to download in SharePoint/OneDrive/Teams or the folder you want to upload to and look at the document path.

The first top-level folder all SharePoint/OneDrive/Teams drives have is "Documents". DO NOT include Documents as part of your folder path and DO NOT start or end the folder path with a forward slash "/". Everything after Documents until the actual file itself becomes your M365_FOLDER_PATH.

The M365_FILENAME variable is the name of the file on SharePoint/OneDrive/Teams in that folder that you want to download, or the name of the local file you wish to upload to that folder. Let me run through three examples below.

This first example would be if you wanted to download some text file within your Teams General folder called test.txt. Your M365_FOLDER_PATH variable would be set to "General" and your M365_FILENAME variable would be "test.txt" to download the file shown below.

This second example is what is shown in sample.env. This is a case where you want to download a file called NetOps Work Tracker.xlsx that is numerous subdirectories deep. Your M365_FOLDER_PATH variable would be set to "Network Operations/On-Call & Scheduled Work" and your M365_FILENAME variable would be "NetOps Work Tracker.xlsx" to download the file shown below.

For this third example, let's say we wanted to upload a file named upload test.txt to the same directory that we used in the second example. To do this, first make sure your file you want to upload is in the same directory as the script. Next, set your M365_FOLDER_PATH variable to "Network Operations/On-Call & Scheduled Work" and your M365_FILENAME variable would be "upload test.txt" to upload the file to the folder above.

After completing this step, you should have five variables filled out in your msal_config.env file; CLIENT_ID, AUTHORITY_URL, M365_DRIVE_ID, M365_FOLDER_PATH, and M365_FILENAME.

The last major setup needed for your msal_config.env file is acquiring an MFA secret for your account. We want to use the MFA secret generated and used for general authenticator apps, but since Microsoft wants you by default to use Microsoft Authenticator, we need to take a few extra steps to get that secret.

First, login to the Microsoft Account Settings page and select UPDATE INFO in the Security info section.

Next, select Add sign-in method.

Select Authenticator app as the method you would like to add.

Click the I want to use a different authenticator app link. After clicking this link, you will be asked to add a new account in your authenticator app of choice, which you can click right through as it doesn't matter for this script.

Next, click the Can't scan image? button and you will finally see your MFA secret key. Copy this secret key into the MFA_SECRET variable in your msal_config.env file.

You will then be asked to enter the six-digit MFA code that correlates to your MFA secret. You can either scan the QR code provided or run the mfa_code_generator.py script to get those six digits and click Next.

Finally, make sure your default sign-in method is set to the newly created Authenticator app or hardware token - code setting. This makes your default login use the six-digit MFA code instead of an app notification or texting a number. The script will not work if any other MFA method is selected for the default sign-in.

After completing this last step, you should have six variables filled out in your msal_config.env file; CLIENT_ID, AUTHORITY_URL, M365_DRIVE_ID, M365_FOLDER_PATH, M365_FILENAME, and MFA_SECRET.

Your last two variables M365_USERNAME and M365_PASSWORD don't require any fancy setup and are pretty self explanatory. M365_USERNAME and M365_PASSWORD are the username and password for your Microsoft 365 account you wish this script to use. The account must have access to the file you want to download or access to the folder you wish to upload to.

This is a very easy step to do. Simply run python3 -m pip install -r requirements.txt -U or if you are on a *nix system, run prerequisites_installer.sh (same command just in a shell script). You may need to specify the python subversion in the command (python3.8 vs just python3) if the above command does not work.

This script uses Firefox headlessly (without a GUI) to login to the provided account and generate a token for the file download. As such, Firefox is required to be installed on the PC the script is running on. A link to Firefox's homepage with a download link can be found here or install it using whatever package manager your *nix system is using (normally apt, rpm, yum, or dpkg).

Additionally, Selenium (the web automation framework being used) requires something called a webdriver in order to control the browser programmatically. These webdrivers are unique to each browser and frequently need to be the same version as the browser in order to work (Although Firefox is much more lax on this compared to Chrome).

Firefox's webdriver is called Geckodriver, and tends to be updated every 3-6 months and will work for all the Firefox versions between updates. You can find your current Firefox version in the GUI by going to the Menu in the top right and navigating to Help > About Firefox, and your version should be in the window that pops up in the center. If you are going the CLI route, you can run the command firefox --version to see the currently installed version.

This platform page maps each geckodriver release to what version of Firefox they will work with. Find what geckodriver release you need and go to this geckodriver download page, scroll down until you see the Assets dropdown for the release you need, and download the geckodriver that corresponds to your operating system.

After downloading it, extract the geckodriver file and put it in the same directory as the script if you are on Windows. If you want, you could add it to your system environment variables, but this isn't necessary on Windows.

If you are on a *nix system, copy/move the extracted geckodriver file to a directory that is in your system PATH, such as /bin, /usr/bin, /usr/local/bin, or others. You can check to see what folders are in your PATH variable on *nix machines by running the command echo $PATH. If you are able to run the command geckodriver anywhere on the machine after copying it into a directory that is in your system PATH, then this step has been completed successfully on.

For some reason, you are unable to just simply put the geckodriver executable in the same folder as the script like you can on Windows.

Listed below are general questions and problems that I either encountered myself or was asked about.