Moving Files with SharePoint REST APIs in Microsoft Flow.md

Moving Files with SharePoint Online REST APIs in Microsoft Flow

About

This is an overview of how to move files with Microsoft Flow using the SharePoint - Send an HTTP request to SharePoint action and SharePoint Online REST APIs.

These SharePoint Online REST APIs can be used outside of Flow, but you will need to handle authentication and provide an X-RequestDigest header with any POST requests (as with any SP REST POST request).

In the case of Flow's SharePoint - Send an HTTP request to SharePoint action, authentication is handled for you and you do not need to provide the X-RequestDigest header.

Why

This is something I came across while building a two-stage approval workflow with Flow, which had the following requirements:

• Allow users to submit a document for approval (using Flow's SharePoint - For a selected item trigger)
• When triggered, the Flow should create a copy of the document in a Stage 1 Approval folder (in the same library) and start an approval process on the approval copy of the document via the Approvals - Start an approval action
  • If approved at stage 1, move the approval copy of the document to a Stage 2 Approval folder (in the same library) and start another approval process
  • If rejected, move the approval copy of the document to a Rejected folder
• If approved at stage 2, move the approval copy of the document to an Approved folder

While Flow already had a SharePoint - Copy file action, there wasn't (at least when I wrote this) a SharePoint - Move file action.

At this point I started looking at using the SharePoint - Send an HTTP request to SharePoint action (the most useful Flow SharePoint action?) with SharePoint Online REST APIs to move files.

Several samples online (including Microsoft documentation) suggest the following API call (or some variation) to move a file:

https://tenant.sharepoint.com/_api/web/getFileByServerRelativeUrl('/Shared Documents/FileName.docx')/moveTo(newurl='/Shared Documents/FolderName/FileName.docx',flags=1)

In the example above, flags=1 means overwrite an existing file at newurl if there is one, rather than returning an error.

This was fine in early development, but with more testing I found the following issues:

• Long file names / paths can quickly exceed URL limits causing the API request to fail
• ' characters in file/folder names are problematic

I spent some time trying to work around these issues, but was asking myself why I was battling with sending POST data in the request URL that should really (in my humble opinion) be included in the POST body instead.

And then I started thinking about how SharePoint Online's modern document library UI has a 'Move To' button...

Inspecting SharePoint API traffic

If you're curious about how SharePoint Online's modern UI uses SharePoint REST APIs, you can inspect the Network traffic sent from the SharePoint page using either the Network tab of your browser developer tools, or a debugging proxy like Telerik's excellent (and free) Fiddler.

This can be a useful technique for discovering as-yet undocumented APIs or how to use APIs that are documented but not explained clearly enough with examples.

In this post I'll focus on using the Network tab of Chrome's developer tools, but the same principals should apply to other browser development tools.

To only show API requests, you'll want to filter traffic shown in the Network tab by clicking on the XHR heading. You can additionally filter by a text value in the 'Filter' text box.

SP.MoveCopyUtil in the modern UI

If you navigate to a SharePoint Online document library that's configured to show the modern UI and (while inspecting browser network traffic) use the 'Move To' button to move a file to a different folder in the same library, you'll see the following REST request:

https://tenant.sharepoint.com/_api/SP.MoveCopyUtil.MoveFileByPath()

This endpoint accepts source and destination URLs in the request body rather than the request URL, which means it doesn't have the drawbacks of the getFileByServerRelativeUrl('/from/')/moveTo(newUrl='/to/') approach. 🎉

If you try and move a file to a folder that already contains a file with that name, the API request will return an error (HTTP 400) and you'll see the prompt A file with this name already exists with an option to replace the existing one. If you click Replace, the request is sent again with an additional parameter:

https://tenant.sharepoint.com/_api/SP.MoveCopyUtil.MoveFileByPath(overwrite=@a1)?@a1=true

If you inspect the request body (in Chrome this is via Headers > Request payload), you'll see something like the following JSON:

{
    "srcPath": {
        "__metadata": {
            "type": "SP.ResourcePath"
        },
        "DecodedUrl": "https://tenant.sharepoint.com/Shared Documents/FileName.docx"
    },
    "destPath": {
        "__metadata": {
            "type": "SP.ResourcePath"
        },
        "DecodedUrl": "https://tenant.sharepoint.com/Shared Documents/FolderName/FileName.docx"
    }
}

As you can see, the DecodedUrl values are absolute URLs that include the source and destination file names.

Recreating the SP.MoveCopyUtil.MoveFileByPath request

You can recreate this request in Flow using the Data Operations - Compose and SharePoint - Send an HTTP request to SharePoint actions, as follows (using the example JSON above as a starting point for the Data Operations - Compose action's Inputs value):

Creating the request body in a separate Data Operations - Compose action makes debugging the Flow easier if something goes wrong with the request.

When configuring the SharePoint - Send an HTTP request to SharePoint action, you should specify the following values:

Properties

Property	Value
Site Address	The SharePoint site URL
Method	POST
Uri	_api/SP.MoveCopyUtil.MoveFileByPath(overwrite=@a1)?@a1=true
Headers	See Headers table below
Body	The Output of the Data Operations - Compose action

Note: swap out the Uri value for _api/SP.MoveCopyUtil.MoveFileByPath() if you don't want to overwrite an existing file in the destination.

Headers

Name	Value
Accept	application/json; odata=nometadata
Content-Type	application/json; odata=verbose

Is that it?

Well, almost. If you only want to move files from one location in a document library to another location in the same document library, that's pretty much the end of the story.

But what about...

Copying files

If you click the Copy To button in the modern library UI while inspecting browser network traffic, you'll see the following request (or a variation thereof):

https://tenant.sharepoint.com/_api/SP.MoveCopyUtil.CopyFileByPath()

As with SP.MoveCopyUtil.MoveFileByPath, you can recreate this request in Flow with the SharePoint - Send an HTTP request to SharePoint action to copy files.

Metadata and version history

Move To - in the same library

When moving files from one location in a library to another location in the same library, the list item associated with the source and destination file is the same, meaning that version history and field values are automatically preserved, as you are only updating the file location within the library.

Move To - in another library, site or site collection

If you use the above approach to move a file from one library to another library, or even to another site or site collection, the list item associated with the source and destination file are not the same (think of it as a 'copy' followed by a 'delete' of the original file) and version history is not maintained.

In this case field values will be replicated from source to destination (if possible).

However, although it's possible to move files across site collections with SP.MoveCopyUtil.MoveFileByPath, if you move a file from one site collection to another via the modern UI's Move To button while looking at your browser's network traffic, you'll notice that it switches to a different set of APIs...

CreateCopyJobs and GetCopyJobProgress

_api/site/CreateCopyJobs

_api/site/GetCopyJobProgress

I've experimented a little with recreating the CreateCopyJobs and GetCopyProgress APIs in Flow, but these probably deserve their own post. So far I have noticed the following from inspecting network traffic while moving files from one site collection to another via the Move To button and recreating the calls from Flow:

• The CreateCopyJobs API will return an error if fields from the source list are missing in the destination list, unless an AllowSchemaMismatch parameter in the request body is specified as true. This is matched by the UI prompting you of the same (missing fields), with a button to proceed anyway, which then resends the request with the AllowSchemaMismatch parameter set to true.
• Version history is copied when using CreateCopyJobs with the IgnoreVersionHistory parameter set to false. If the parameter is set to true, the destination file will be version 1.0.
• If the destination file already exists when the IsMoveMode parameter is set to true, the copy job will fail (see the Logs when inspecting the output of GetCopyJobProgress). There doesn't appear to be an 'overwrite' parameter in the CopyMigrationOptions class to allow overwrite on a move. This also happens with the UI Move To button - the following error message is displayed: A file or folder with this name already exists. Please rename the file or folder and try again.

API references for CreateCopyJobs and GetCopyJobsProgress

• CreateCopyJobs
• GetCopyJobsProgress
• CopyJobProgress
• MigrationJobState enum

Hopefully someone finds the above useful 🙂

Updates

Date	Update
22nd June 2018	Updated to reference 'SharePoint Online' rather than 'SharePoint', as the post content has only been tested in that context
22nd June 2018	Updated info on CreateCopyJobs and GetCopyJobProgress based on further testing

Hi @zplume,
I was able to resolve the issue by doing a "Create File" action, using the meta data from the original file, instead of a "Copy File" action.
Thanks for all of your assistance.

@amonjohnson - in my testing (looking at the Flow run history) the Folder path property of the Get file properties action output includes both the folder and the parent library name ("HT" in my test) and I'm assuming you don't want the library URL in your destination.

"HT/SourceFolder/"

If you add a Get folder metadata using path action and pass in the Folder path returned from Get file properties, you can retrieve the folder Name without the library URL.

I was able to use the Name property returned from Get folder metadata using path in the Destination Folder input of the Copy file action to copy a file to that folder in the destination library (assuming the folder already exists there).

You'd need to do a bit more work (perhaps with expressions) for handling nested folders in the source location.

Hi @zplume,

We have a similar setup - slight difference in that we use a field to manage publishing with a scheduled flow that looks for items to move to 'Publish' folders (Staging [pending approval] -> Published to Dpmt OR Published to All)

So for the 'Published to' folders we have it set with unique permissions for the relevant groups.
A problem that has been raised is where the item has been shared initially then gets moved. Apparently when moved (the flow uses SP.MoveCopyUtil.MoveFileByPath) the file retains the unique permissions.

Any ideas on how to merge or inherit the permissions from the destination folder?
Ideally looking for some parameter or payload for the API request but I appreciate I may need to add an additional step before/after the move.

Hi,

Does flow have restictions moving large files (1GB + )?

Hi,
does anyone know why it is not possible to move a file from a Document Library to another Document Library? I mean without the workaround of copying original file, pasting it in the destination location and, finally, deleting the original one, in a way to save, for example, history version and others metadata or shared urls.

Thanks in advance.

Hi @DTGitRepository, I just saw your message - I appreciate this is a bit late, but if you're still looking for a way to reset permissions for a file/item in SharePoint via Flow, give me a shout on Twitter: https://twitter.com/lsspdev

Hi @kruismanp, SharePoint itself should be able to handle moving large files I think (e.g. via the SharePoint document library UI).

I haven't done any testing around various APIs / Flow actions to compare their handling of large files - if you look into this yourself, I'd be interested to know your findings!

Hi @SirLac, have you seen my followup post on CreateCopyJobs?
https://gist.github.com/zplume/21248c3a8a5f840a366722442cf9ee97

The API is a little more complex to use but did copy version history in my testing (see post linked above for details).
It might be worth testing the SharePoint 'Copy file' and 'Move file' actions in Flow to see if either of these replicates the version history in the destination item.

Hi @zplume, great post - thanks for sharing.
If anyone is wondering how to move a file between folders and "Keep Both" files, which renames the moved file by appending a 1 to the filename, the following worked for me:

Compose step

{
    "srcPath": {
        "__metadata": {
            "type": "SP.ResourcePath"
        },
        "DecodedUrl": "Server/Lib/SourceFolder/Doc.doc"
    },
    "destPath": {
        "__metadata": {
            "type": "SP.ResourcePath"
        },
        "DecodedUrl": "Server/Lib/DestFolder/Doc.doc"
    },
    "options": {
        "__metadata": {
            "type": "SP.MoveCopyOptions"
        },
        "KeepBoth": true,
        "ResetAuthorAndCreatedOnCopy": false,
        "ShouldBypassSharedLocks": true
    }
}

HTTP request step
Uri is just _api/SP.MoveCopyUtil.MoveFileByPath()

Hi there

Im struggling with the last steps to move an item list to another folder... Within my flow, the HTTP function takes a along time to process (came up as bad gateway)... Is there any reason why?

The list items which I am hoping to move will have attachments which will be no more than 25 - 30MB.

Your assistance in this matter is highly appreciated!

Finally got it to work.....

Hi @zplume, Firstly Thanks for sharing this wonderful blog, but just wanted to check if there is a way to move or copy Multiple Folder's/File's from one folder to another folder ?

I tried using this example to move files from Sharepoint Online to Sharepoint On-prem, but without success.
Do you know how to achieve it with Power Automate?

Hi @SirLac, have you seen my followup post on CreateCopyJobs? https://gist.github.com/zplume/21248c3a8a5f840a366722442cf9ee97

SharePoint & OneDrive Copy and Move API replaces the CreateCopyJob API. The CreateCopyJob API has been deprecated.

The parameter exportObjectUris does not exist in method CopyFileByPath