SharePoint Online site provisioning using Microsoft Flow, Azure Functions and Azure Storage Queue

A while back I created a provider hosted app using CSOM in C# for creating project sites but this required the users to have sufficient permissions to create a site. Using Microsoft Flow, Azure Function, Azure Storage Queue, PowerShell and SharePoint Online I created a proof of concept with the latest techniques and using the AppId/AppSecret so the user doesn’t need additional permissions. This solution isn’t free as it needs an Azure Subscription but the costs are minimal. Please find references to Microsoft in the summary at the end.

This article describes the following scenario:

  1. The user creates an item in a SharePoint list.
  2. Microsoft Flow will be triggered on item creation.
  3. Microsoft Flow will add a message on the Azure Storage Queue.
  4. The Azure Function will monitor the Azure Storage Queue and create the subsite based on the values entered in the SharePoint list using PowerShell.

This article has the following chapters:

  • Create SharePoint List
  • Get and register AppId and AppSecret in SharePoint Online
  • Create Azure Storage Queue
  • Create Azure Function
  • Create PowerShell Script
  • Test Azure Storage Queue
  • Create Microsoft Flow

Create SharePoint List

First we are going to create a list in SharePoint which we are going to use for our site metadata.

SNAGHTML1b92ddf2
Add an App

image
Custom List

image
Create

image
Add the below columns:

  • SiteURL –> Single line of Text
  • SiteTemplate –> Choice
  • SiteLanguage –> Choice

image

The list has been created which we are going to use for our site provisioning.

Get AppId and AppSecret in SharePoint Online

It is possible to use a username and password for the Azure Function but it is also possible to use an AppId and AppSecret for impersonation.
In this scenario we are going to use an AppId and AppSecret.

Go to the site collection where you want to register the app by appending the url with “_layouts/15/appregnew.aspx”

image
Fill in the above information and click on create

image

Save the Client Id and Secret as we are going to need it for our Azure Function.
Next append /_layouts/appinv.aspx to the url

image
With the below Permission Request XML we allow the app access to the site collection. You can specify different levels which are explained at https://docs.microsoft.com/en-us/sharepoint/dev/sp-add-ins/add-in-permissions-in-sharepoint .

<AppPermissionRequests AllowAppOnlyPolicy=”true”>

<AppPermissionRequest Scope=”http://sharepoint/content/sitecollection” Right=”FullControl” />

</AppPermissionRequests>

and click on Create

image
Trust It

Create Azure Storage Queue

We are going to setup the Azure Storage Queue which will handle all our messages which have been sent using Microsoft Flow.
Please note that this can also be achieved without the Azure Storage Queue as you can directly sent the message to the Azure Function using an Azure HttpTrigger function.

First go to your Azure Dashboard

image
Storage accounts

image
Add

image
Create

image
Open the newly created storage account

image
Click on Queues

image
+ Queue

SNAGHTML1bad6cd5
OK

The Azure Storage Queue has now been created which we use within our Microsoft Flow and Azure Function.

Create Azure Function

The next thing we will build is the Azure Function. The Azure Function will be created based on PowerShell and the SharePointPnPPowerShellOnline module.
We are going to start from the Azure Dashboard.

image
Go to the App Services

image
Add

image
Function App

image
Create

image
We are going to use the existing resource group and storage which we created during the Azure Storage Account. Click on Create

image
Open the newly created Azure Function

image
New function

SNAGHTML1bcaa56d
Enable Experimental Language Support and navigate to Queue trigger

image
Click on PowerShell

SNAGHTML1bcdd008
Enter the queue name we created earlier. And click on New

image
Select the Azure Storage Account

image
Create and navigate back to the Platform features

image
Go to Platform features

image
Open Advanced tools (Kudu)

image
Click on Debug Console and then on PowerShell

image
Navigate to Site –> wwwroot –> QueueTriggerPowerShell

image
Create a new folder called “modules”

image
We are going to upload the PowerShell DLL’s which we are going to use here as it is not possible to import-modules from within the Azure Function. You can drag and drop the files to this folder.
The files we need are by default installed in the following location: C:\Program Files\WindowsPowerShell\Modules\SharePointPnPPowerShellOnline

image
Copy the contents from this folder to the Azure Function.
If you are missing this folder; Install this using PowerShell on the workstation with the command: Install-Module SharePointPnPPowerShellOnline

image

Also copy the items from the following locations:

  • C:\Windows\assembly\GAC_MSIL\Microsoft.IdentityModel
  • C:\Windows\Microsoft.NET\assembly\GAC_MSIL\Microsoft.IdentityModel.Extensions

Go back to the function

image
Go to the application settings

image
Select 64-bit and scroll down

image
Add the AppId and AppSecret with the key to the application settings as we can reference to these settings from the Azure Function.
Save the modification and in the next chapter we will create the PowerShell script.

Create PowerShell Script

Go to the QueueTriggerPowerShell in the Azure Function

image

image
Add the below PowerShell code

$requestBody = Get-Content $triggerInput -Raw | ConvertFrom-Json

$ParentSiteUrl = "https://spfire.sharepoint.com/sites/projectsitecreation/"

$WebTemplate = $requestBody.WebTemplate

$SiteTitle = $requestBody.SiteTitle

$SiteDescription = "Site with PowerShell"

$SiteURL = $requestBody.SiteURL

$SiteLanguage = $requestBody.SiteLanguage

$AppId = $env:AppId

$AppSecret = $env:AppSecret

connect-PnPOnline -AppId $AppId -AppSecret $AppSecret -url $ParentSiteUrl

New-PnPWeb -Title $SiteTitle -url $SiteURL -Locale $SiteLanguage -Template $WebTemplate -Description $SiteDescription

Write-Output "PowerShell script processed queue message '$requestBody'"

image
Click on test in the right corner

image
Enter the below Request body

{
“WebTemplate”: “STS#0”,
“SiteTitle”: “TestCreation1”,
“SiteURL”: “TestCreation1”,
“SiteLanguage”: 1033
}

and click on Save and run

image
You can verify the log for success and navigate to the created site

SNAGHTML2a2781
We now know that the PowerShell code is successful.

Test Azure Storage Queue

Go to the Azure Storage Queue to test if adding a message is being successfully processed by the Azure Function.

image
Add message

image
OK and you can verify if the Azure function picked up the message if you still have the log open

image
Or go to the newly created site

SNAGHTML302e3a
We confirmed the Azure Storage Queue with the Azure Function is working correctly.

Create Microsoft Flow

We can now create a Microsoft Flow that will add an message in the Azure Storage Queue which will be picked up by our Azure Function.
Go to https://flow.microsoft.com

image
Create from blank

image
When an item is created

image
And add a new step

image
Put a message on a queue

image
Add a new connection if you already had one like me

image
The Connection Name can be anything where the Storage Account Name and Shared Storage Key can be found in Azure

SNAGHTML5123e9

image

Save the flow and create a new item in the previous created SharePoint List

image

Save and first verify the Microsoft Flow

image
Next verify the Azure Function Log if still open

image
And last verify if the site has been created

SNAGHTML59251b

The site has been created successfully.

Summary

We have now created a working site provisioning solution based on a SharePoint list.
This solution uses multiple techniques such as Microsoft Flow, Azure Storage Queues, Azure Functions and SharePoint Online.
This is just an example of working with these techniques but you can for example do more after the site creation such as adding extra permissions and set default columns.
It is possible to do more with Microsoft Flow as for example send an email after creation or update the status during the creation

You can find more information at https://docs.microsoft.com/en-us/sharepoint/dev/declarative-customization/site-design-pnp-provisioning regarding for example an app ID and app secret with administrative rights on your tenant, Microsoft Flow and an Azure function. Costs for an Azure Function are mentioned in https://azure.microsoft.com/en-us/pricing/details/functions and queue costs at https://azure.microsoft.com/en-us/pricing/details/storage/queues/

Information about the SharePoint PnP PowerShell CmdLets can be found at https://github.com/SharePoint/PnP-PowerShell and https://docs.microsoft.com/en-us/powershell/sharepoint/sharepoint-pnp/sharepoint-pnp-cmdlets?view=sharepoint-ps

Please let me know your use case for Azure Functions and if there are any questions.

PowerShell Office 365 Inventory tool

I’ve thought of creating a different tool after creating the PowerShell Office 365 tool a couple of days ago to improve my PowerShell coding and to make my work and that of my colleagues a little bit easier. The PowerShell Office 365 inventory tool lists a lot of information you would like to see when troubleshooting or getting information from a tenant you never connected to.

You can follow and download the PowerShell Office 365 Inventory Tool at GitHub: https://github.com/peetersm12/Office365Inventory-GUI

This PowerShell Office 365 Inventory tool will retrieve the following information:

  • Azure Active Directory Users
  • Azure Active Directory Deleted Users
  • Azure Active Directory External Users
  • Azure Active Directory Contacts
  • Azure Active Directory Groups
  • Azure Active Directory Licenses
  • Azure Active DIrectory Domains
  • Exchange Mailboxes
  • Exchange Archives
  • Exchange Groups
  • SharePoint Sites
  • SharePoint Webs

Please note that you will need a few pre-requisites before fully able to run this tool:

Please let me know which information you would like to see added to this tool and I’ll add this is as soon as possible.

FrontWindow2

[How to] PowerShell Office 365 Inventory Tool

image

Run ‘Start-Office365Inventory.ps1’. You will automatically be asked to run as administrator if you ran this as a normal user.

image

First Connect to Office 365 by clicking on the credential logo, by pressing f4 or via the menu

image
Fill in your credentials and press OK

image

You are connecting successfully when the icons are green.

There are 2 possible actions now:

  • Navigate to a tab and run only this action
  • Press the run all button to run all available actions

image

Only the Azure Active Directory Licenses have been returned. You can verify which actions have been run on the home tab.

image

Now click on the run all actions button. You can see the progress on the Home tab.

Please note that you will need site collections permissions for the SharePoint Webs option as this uses CSOM to connect to the different webs.

You will receive the following message but it will continue looking for more webs

image

After all actions everything should be green

image

Run the action individually if you encounter an error as the exception will be shown in the below message center.

Reports
You can create a .CSV or .HTML file based on the items visible at any time in the view by clicking on the button next to the CSV Report.
The report will be saved to the report folder located in the script root.

The .CSV report will look like:
report1

The .HTML report will look like:
report2

It is also possible to create a .HTML file for all the available tabs in a nice format. Please note that this is still a work in progress but it will lists everything at the moment.

report3

Error log
Some error information is displayed on the background PowerShell window as other information is only readable using the errorlog.
Please send me this information and the message in the below message box if you encounter an error.

image

Several useful PowerShell cmdlets for Office 365

On 22-11-2016 I presented a presentation with the title “Manage Office365 quick, painless and safe with PowerShell” at Experts Live 2016. During this presentation I showed several useful PowerShell cmdlets for Office365 and mainly for the Azure Active Directory, Exchange Online, SharePoint Online and Office365 Groups.

EXPERTSLIVE.5011_email-signature_spreker_ENG_630x180

The cmdlets presented can be downloaded as a .zip file using the below download button including the script to retrieve information from Office365 and the presentation in Dutch. A couple of cmdlets are also added below from the .ps1 file.

Start transcript

A best practice is to start a transcript of the cmdlets being entered in PowerShell and the corresponding output. Use the following one-liner to start the transcript to the specific folder

Start-Transcript -Path "C:\Users\mpeeters\OneDrive - Valid\Valid documents\My Transcripts\PS_$((get-date).ToString("ddMMyyyy")).txt" –append

Azure Active Directory cmdlets

The Azure Active Directory module has to be updated if using for example the 1.0.8070.2 version. You can check the version using the below cmdlet and download the newest version at Technet

(Get-item C:\Windows\System32\WindowsPowerShell\v1.0\Modules\MSOnline\Microsoft.Online.Administration.Automation.PSModule.dll).VersionInfo.FileVersion

The next cmdlet can be used to get all external users currently in the Office365 tenant.

Get-MsolUser -all | Sort -Property SignInName | where{$_.UserPrincipalName -like "*#ext#*"} | select SignInName, UserPrincipalName, DisplayName, WhenCreated

image

Use the following cmdlet to get all the external users which have a mismatch between SignInName and UserPrincipalName

Get-MsolUser -all | Sort -Property SignInName | where{$_.UserPrincipalName -like "*#ext#*" -and $_.UserPrincipalName -notlike "$($_.SignInName.split("@")[0])*"} | select SignInName, UserPrincipalName, DisplayName, WhenCreated

image

Exchange Online

First connect to Exchange Online using a remote PowerShell session


$UserCredential = Get-Credential
$EOSession = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri 
https://outlook.office365.com/powershell-liveid/ -Credential $UserCredential -Authentication Basic –AllowRedirection
Import-PSSession $EOSession -WarningAction:SilentlyContinue 

and use the following cmdlet to grant full control to a user on a certain mailbox where AutoMapping is false

Add-MailboxPermission -Identity "LiveDemo.stark" 
-User mpadmin -AccessRights FullAccess -Automapping $false

and the following for send-as permissions

Add-RecipientPermission -identity "LiveDemo.stark" -Trustee mpadmin -AccessRights SendAs -Confirm:$false

It is also possible to add direct permissions an a users calendar. Keep in mind that this is using the language the user has configured.

Add-MailboxFolderPermission -Identity LiveDemo.stark@spfire.nl:\calendar -user jon.snow@spfire.nl -AccessRights Editor

The following cmdlet will enable archiving for each users mailbox

Get-Mailbox -Filter {ArchiveStatus -Eq "None" -AND RecipientTypeDetails -eq "UserMailbox"} | Enable-Mailbox -Archive

SharePoint Online

We have gotten the external users in the above cmdlet but we can also use the following cmdlet to get the external users on a specific SharePoint site collection

Get-SPOUser https://spfiredev.sharepoint.com | where{$_.loginname -like "*#ext#*"}

Using the next cmdlet will list all available users in the user information list for each available site collection

get-sposite | %{$site = $_.url; Get-SPOUser -Site $site | select @{Name="URL"; Expression = {$site}}, DisplayName, LoginName} | Format-table -AutoSize

image

 

The default storage for OneDrive for Business Online is 1TB but this value can be increased or decreased using PowerShell. It depends on the users license if they can use for example 5TB or more.

Use the following cmdlet to set the OneDrive Storage Quota to 2TB

Set-SPOTenant -OneDriveStorageQuota 2097152

The above cmdlet is for all users but you can also change the value for an individual user with the following cmdlet

Set-SPOSite -Identity https://spfiredev-my.sharepoint.com/personal/mpadmin_spfire_nl -StorageQuota 5242880

The quota can also be reset to the specified OneDrive storage quota

Set-SPOSite -Identity https://spfiredev-my.sharepoint.com/personal/mpadmin_spfire_nl -StorageQuotaReset

Office365 Groups

Office365 groups can currently only be managed using PowerShell but this will soon change.

Use the following cmdlets to check if the mailbox or the document library has been used in the previous 7 days.

Get-UnifiedGroup | Foreach-Object { Get-MailboxStatistics -Identity $_.Identity } | Where-Object {$_.LastLogonTime&amp;nbsp; -ge (Get-Date).AddDays(-7)}
Get-UnifiedGroup | Foreach-Object {Get-SPOSite -Identity $_.SharePointDocumentsUrl.replace("/Gedeelde&amp;nbsp; documenten", "")} | FT Title, Url, LastContentModifiedDate, ResourceUsageCurrent

The document library is bound to the tenant language and keep in mind that there are two spaced if using the dutch language. In English this will be “Shared Documents”

Disable the creation of groups for all users with the following cmdlet

Set-OwaMailboxPolicy -Identity spfire.com\OwaMailboxPolicy-Default -GroupCreationEnabled $false

and the following for just 1 or more users

New-OwaMailboxPolicy -Name "LiveDemoDenyGroupCreation"
Set-OwaMailboxPolicy –Identity "LiveDemoDenyGroupCreation" –GroupCreationEnabled $false 
Set-CASMailbox –Identity LiveDemo.stark -OWAMailboxPolicy "LiveDemoDenyGroupCreation"

Each Office365 group will get its own mailbox and also an entry in the Global Address List (GAL). Use the following cmdlet to prevent the Office365 group to be displayed in the GAL

Set-UnifiedGroup -Identity bouwersgroep -HiddenFromAddressListsEnabled $true

Get Everything

During the presentation I also showed a script where I got a lot of information from Office365 and put this information in an Excel file.

This script is also located in the download and change the transcript and output location before using the script.

image

Download the .ps1 files and presentation

The above cmdlets are just a few which are present in the .ps1 file used during the presentation. Please let me know in a comment if these files were helpful and how you used some cmdlets.