Lightning Talks are Back!

Every December, we at the Tri-State Office 365 User Group like to end the year with Lightning talks. Lightning Talks are 2 hours of rapid presentations. Each presentation is roughly 15 minutes long and they’re typically demos of cool things that people have worked on. In the past, we’ve seen demos showing how to provision sites using templates in SharePoint, SPFx development tips and tricks, a Power App that was used to catalog personal media, and other cool demos.

This is a great time to get involved in the community. If you have a cool demo to show, if you have an interesting use case, feel free to reach out and submit a topic.

If you are someone who has been interested in presenting but haven’t quite taken that step, what perfect way to do it than a short demo in a supportive environment. So if you are interested in participating on December 8, 2020 and have a topic that you want to share, join the meetup and use the contact the organizer link.

4 SharePoint Features that will Help You Reorganize Your Intranet

Do you still have an old, empty or near empty Team site as the homepage that people visit when they go to SharePoint? Has your organization attempted to use SharePoint as something other than a dumping ground for files but can’t seem to quite get there? There’s probably some months/years old company news on the homepage, a few links that most people have bookmarked, maybe a calendar with no recent events. In most cases, organizations don’t put in the proper time to plan for how they want to use SharePoint and there isn’t clear ownership. Eventually, SharePoint provides little value to your business because you haven’t properly tended to it. The overall fix is a topic for another conversation (or rant) but in this post, I’m going to offer up some suggestions for cleaning up your environment. Ultimately, governance, training, and adoption planning will need to be addressed but here are some features that will help you get your house in order.

1. SharePoint Home Sites: Your Landing Page

The landing page is the first impression. If it’s largely blank, has stale content, a handful of links, an empty calendar, a navigation menu with links that aren’t relevant/useful to most employees, how can you expect people to visit it? You probably don’t know where to even begin and for that, I suggest the SharePoint look book for inspiration or just use one of their templates.

Look book templates

For this example, I chose to provision a new site using “The Landing” template (2nd column, 2nd row in the image above) and my url is https://<tenant>/sites/TheLanding. It looks something like the image below. Instantly better than a typical, neglected homepage, right?

The Landing layout

The great thing about these templates is that it takes a lot of the planning and second guessing out of your hands. It’s not completely a set and forget solution though. If you don’t use Yammer, then you don’t really want an empty Yammer feed on the page, right? The templates are a great starting point but you need to have someone own or maintain it. It’s great to have content automatically surface for the visitor like the My Recommendations section, frequent sites, and my recent documents, but someone needs to provide news or material that makes it worth the visit. One suggestion would be to stop sending HR communications or messages from the CxO via email and create a news article that people can read or look up. I know I would be grateful for less email clutter!

Setting a New Home Site

Once the above is ready to go live, you can you can designate it as your new home site by using the following PowerShell command:

Set-SPOHomeSite -HomeSiteUrl <your new site url> 

NOTE: This does NOT replace your root site

So if it doesn’t replace the root site, what does setting this site as a home site do for us? I’m glad you asked.

For starters, the site’s search scope changes from searching only itself to searching all sites.

Before becoming a home site
After becoming a home site

The site is also set as an Organization News Site which is a site that is flagged an an authoritative site. The result is that your news articles get highlighted. In this example, you can see an article published on The Landing and another on the Support site but only The Landing’s article is set as the authoritative site.

If you want to set other sites as Organization News Sites, you can use the following powershell command.

Set-SPOOrgNewsSite -OrgNewsSiteUrl <site url> 

Another thing that happens when you set a Home Site can be seen on the mobile app. The home button on the mobile site will take you to the new Home Site (even though you don’t see the same in the browser).

Swapping Root Sites

I mentioned that changing the Home Site doesn’t change your root site. If you visit https://<tenant&gt;.sharepoint.com, whatever was there before will still be there now but it’s simple to change that these days. Simply go to your Admin Center, select your current root site from the list of Active Sites, click the Replace site button, and provide the URL for the new site collection.

It’s actually a good thing that it doesn’t change right away. It allows you to test it out before fully releasing although, it does immediately change the experience from the mobile app. Let’s face it, if your site is stale and you’re looking for a quick way to bring value to it, then your users are probably not using the app either.

2. Navigation

Another area that people find challenging is navigation. I typically see the problem being that people find it difficult to figure out what to provide links for or they want to provide links for everything. When I talk to customers, I tend to talk about the intranet and the collaboration areas as separate things where the intranet is more of a traditional communication vehicle with few people updating content and collaboration areas being more adhoc or less uniform than the other sites. There’s usually no good reason to have a traditional intranet with a link to a project site. You end up cluttering your navigation. Microsoft Teams is making that conversation easier to have because it seems to make that distinction clearer.

If you treat your intranet as a somewhat locked down communication tool, then you reduce the number of potential sites/pages that you want to navigate to. The natural thing that most do is create a link for each department but who says that’s what you need to do? Creating department links on an intranet implies that you will have similar communication sites for each department and from my experience, those sites are left to the departments to manage and most if not all won’t maintain them. If you insist on having links for departments, keep it simple. In fact, maybe don’t even create a site. Instead, just create a site page that your communications team maintains. Departments can submit articles to be posted but communications can approve them.

My opinion, I feel like if you have a link in the top nav, it should take you to a site that looks and feels similar. However, if you have a link in the body of the page, they feel more like links to “other” resources. With that said, I would use the Sites web part to show links to sites that the current user frequents.

Don’t Clutter you Nav with Collaboration Sites

Expanding on the previous thoughts on intranet vs collaboration and what they should look like… Places where people go to author content for projects or small teams are spaces that I like to keep separate from the intranet. Project sites where only a select few need access are areas that, for me, don’t need to have a dedicated link from the intranet. It can show up in the Sites web part like the screenshot above but if someone is maintaining a list of links in the top nav, or a mega menu, or anywhere for that matter, I wouldn’t. You don’t want to have to manually clean up a bunch of stale links when projects end.

I also feel that these types of sites shouldn’t have to follow the corporate standards for branding. Departments or project teams should be able to manage their own content however they feel works best. If one department likes to use search and metadata while another department prefers to use a hierarchy of folders, they should be allowed to do so. That’s the common comparison but there are others. Some businesses like to impose a specific folder structure in a library and it may not work well with a team so they just create another library and set it up their way anyway.

What if you have dozens of sites that have loose relationships with each other like project sites for a specific customer or even project sites related to a specific department? In those cases, I would consider leveraging Hub Sites.

3. Hub Sites

Hub Sites is a feature was released a little over 2 years ago. Hub Sites are a way to bolt sites together and allow those sites to share a navigation and theme. It also scopes your search to return results from sites associated with the hub and you can bubble up content from the associated sites up to that hub. In the days of subsites, if you decide to move a subsite, it was a bit of a pain because you had to make sure permissions and content came over. Now, you pick a hub that you want to join and you’re done. If you go through department reorgs or acquire another business, hub sites can simplify the grouping of those sites.

Register a Hub Site

In order to group these sites, you just need to select the site in the Admin Center and register it as a Hub Site.

Associate with a Hub Site

Once a site is registered, you can associate related sites to that Hub via the Site Information for the site being associated.

4. Audience Targeting

Another useful feature for organizing your site is audience targeting which allows you to show content to a select group. This is not a security feature but I used to see businesses use it that way. It is simply a way to ensure that relevant content is being displayed.

Audience targeting can be enabled on navigation links (image below), site pages, and certain web parts like the News and Highlighted Content web parts. You do have to be careful when using this feature. If you tag the wrong group or use it too much, it will cause confusion as site visitors may be content that they’re expecting to see or the opposite.

Honorable Mentions

While the above is not a comprehensive list, I think they’re a good starting point. There are other features that are super helpful if you have the organization and team to get you there but can sometimes seem daunting for businesses that are looking for quick wins.

Content Types

A really important feature is Content Types and that is a way to classify documents that you are working with. By default, the Content Type used in a library is “Document.” You can create additional content types with their own columns for a particular type of document. An example could be an Invoice with columns for Service Offered, Invoice Number, and Invoice Date which can be used to search or filter for specific files. You can then use those columns to create different views or generate better search results.

You ask anyone who’s worked with SharePoint and that is one thing that they’ll always tell you to use. It’s also something that businesses seem to struggle with or ignore. You get into what content do you commonly use, then what metadata is important, then what metadata can be shared, should we use the term store, who maintains the terms, how do we know when to use them, etc. If that’s you, know that in the long run, it will help you provide value but if you need a quick win, there’s a chance it won’t be front and center for you.

Search

Another important feature is Search. I didn’t place this in the 4 for a couple of reasons. For starters, this can sometimes become a project too. Ensuring that results are relevant and content types are created to help surface the right content. You get search out of the box to begin with so it’s not that you won’t have search. Your results just might not be too great if you don’t take the time to plan and organize your site.

I definitely think you should look at ways to improve your search but what this article has been driving at is putting the most important things front and center on that home page and providing content (news) in a central place to jump start your intranet. So when I want to read about what’s new in the organization, who the new hires are, who got promoted, or resources/files that are frequently needed, I should be able to go to our intranet to find that information. If I go to a project site for Client ABC and their related projects are joined to a hub, I should be able to work my way to those sites and find the relevant content. In those cases, search will already be narrowed to those sites.

Conclusion

I want to make clear that the above items are tools that I think can get you to show something respectable. These are my thoughts on a bare bones solution for places that have struggled to get started. While there are other features that can also help, the ones listed here are a good start to get your site back in order. It doesn’t change the fact that you need to plan, learn, and get people on board to make sure it continues to succeed but sometimes you need a jump start to get you in that direction. In other words, you can build an MVP intranet but you need to plan for incremental improvements.

Join me at M365 Chicago Virtual

On November 13th, I’ll be speaking at M365 Chicago. If you haven’t caught my Intro to AI with Azure and Office 365 session, come and check it out. #M365Chicago is looking to be a huge event with over 80 speakers! If there’s a topic of interest to you, there’s no way it’s not being covered at this event.

If you want to catch my event, it’s currently scheduled to be at 2pm but the schedule hasn’t been finalized. This you can expect to see in my session:

So, be there! This is one of the largest SPSEvents of the year.

Be sure to register for the event!

A Trio of Global Microsoft 365 Developer Bootcamps

For those of you who aren’t familiar with it, the Global Microsoft 365 Developer Bootcamp is a free, one-day training event led by Microsoft MVPs with the support of Microsoft and local community leaders. On November 7, 14, and 21, the Pennsylvania, New Jersey and New York Global Microsoft 365 Developer Bootcamps are back. The events will be virtual and tickets are limited. On each day, we will be doing hands on sessions.

On November 7th, we will be doing a workshop on automating and securing the provisioning of Microsoft Teams. You will learn about Teams Governance, automation, guest access, approvals, and more.

November 14th is Power Platform day. On this day, you will be learning about the different apps that make up the Power Platform and we will guide you through installing one of the samples apps. Additionally, we will get to see a demo of Power Virtual Agents for chat bot creation and we’ll show you how to use AI to automatically extract text from PDF files.

November 21st will be all about the SharePoint Framework. On this day, we will walk you through what the SharePoint Framework is and how to create your first web part.

We will be publishing the details on Eventbrite soon so stay tuned! Since the events are hands on, the team will be available to answer questions and help troubleshoot issues and as a result, tickets are limited for this virtual event.

If you’re a local business that specializes in the technologies above and are interested in sponsoring the event, email contact@spsphilly.org for more information.

I will be working these events with Tom Daly, Manpreet Singh, Michael Mukalian, and James Novak.

Join me at Collab365 GlobalCon4

On October 20th, I’ll be doing a short, 15 minute presentation on AI Builder at Collab365 GlobalCon4. The Collab365 team sure knows how to organize an event and I’m excited to be involved. If you caught my session, “Intro to AI with Azure and Office 365” at GlobalCon3, you saw me walk through several AI based solutions. The goal was to show how simple it can be to incorporate AI using Azure Cognitive Services without having to understand machine learning.

No alternative text description for this image

At GlobalCon3, I walked the audience through several examples including:

It was wall to wall demos but they were surface level demos. I showed the construction but didn’t spend too much time on it. This time, I’m doing a turbo session and showing EXACTLY how simple these solutions can be. I’ll be recreating the AI Builder demo where I extracted text from invoices and updated the file metadata with that extracted text and I will be doing all of that in 15 minutes (I hope). No code, no machine learning. Just AI Builder, a SharePoint document library, and Power Automate. So join me at GlobalCon4, and be sure to check out the other awesome presentations at the event.

Using NVM and Ubuntu on Windows to Manage Node Version Collisions

A few nights ago, I sat in on a PnP Sharing is Caring call with David Warner II and Hugo Bernier. The topic was using Node Version Manager (NVM) to handle different node versions on the same machine. I’m on a Windows machine so we talked about Node Version Manager for Windows since the original NVM was not written for Windows. The thing you find out right away when trying to use NVM for Windows is that it’s a separate project from NVM and it doesn’t have all the same features. In this post, I’ll talk about the 1 missing feature that I really wanted to use, and what I did to get it. The hint is in the title of this post.

For SPFx development, I use Docker for isolation. I had issues setting up the latest version of SPFx in Docker, but NVM seemed to be a great option. In our conversation, we talked about the idea of isolating by customer. As an example, let’s say I have a customer, Contoso, who wants to use Node v14.4.0 and Gulp.js while my other customer, Fabrikam, wants to use Node v14.4.0 and Grunt.js. One way to separate the two would be to dedicate a minor version to a customer. For example, Contoso would run Node v14.4.0 while Fabrikam runs on Node v14.3.0. Maybe you are inheriting a project that is running on a specific version of Node, then the above isn’t an option but you really want to keep the customer specific modules separate.

Note: v14.4.0 is used just for the purpose of setting up NPM and is not a supported version for SPFx development. For a list of node versions that are compatible with SPFX, take a look at the SPFx Compatibility Matrix,

I looked into the available commands for NVM and NVM for Windows and immediately noticed that NVM has more. I saw that they had an option for creating an Alias which NVM for Windows did not. Alias allows you to refer to a version by a name of your choice. What I want to be able to do is name each version by customer. Examples:

  • nvm use Contoso
  • nvm use Fabrikam

The Setup

On the NVM git repo, you’ll see the following info about the tools:

nvm is a version manager for node.js, designed to be installed per-user, and invoked per-shell. nvm works on any POSIX-compliant shell (sh, dash, ksh, zsh, bash), in particular on these platforms: unix, macOS, and windows WSL.

I decided to enable Windows Subsystem for Linux (WSL) which you can do through the Windows Features. This should allow me to run the original NVM tool inside of Windows without having to use the limited Windows version of the tool.

Windows Features 
Turn Windows features on or off 
o 
To turn a feature on, select its check box. To turn a feature off, clear its check 
box. A filled box means that only part of the feature is turned on. 
Telnet Client 
Client 
Virtual Machine Platform 
Windows Defender Application Guard 
Windows Hyper-visor Platform 
Windows Identity Foundation 3.5 
Windows PowerSheII 2.0 
Windows Process Activation Service 
Windows projected File System 
Windows Sandbox 
Windows TIFF IFiIter 
Work Folders Client 
Cancel

Next, you need a Linux environment so I went to the Windows Store and did a search for Linux.

Microsoft Store 
Home Gaming Entertainment 
Results for: Linux 
Departments 
All departments 
Apps (116) 
Show all 
Productivity 
Deals 
Anexinet Corp. 
Ubuntu 
231 
Free 
KALI 
BY OFFENSIVE SECURITY 
Kali Linux 
195 
Free 
Available on 
Ubuntu 20.04 LTS 
42 
Free 
Ubuntu 18.04 LTS 
145 
Free 
susE Linux 
Enterprise Server 
Free 
p Search 
Debian 
125 
Free

I went with Ubuntu and installed it. It was roughly half a gig in size. After installing Ubuntu, you’ll need to restart your machine.

M icrosoft Store 
x 
Gaming 
Entertainment 
Productivity 
Ubuntu 
Anexinet Corp. 
p Search 
ubuntu0 
EVERYONE 
ESR g 
Canonical Group Limited • Developer tools > Utilities 
231 Share 
Ubuntu on Windows allows you to use Ubuntu Terminal and run Ubuntu command line utilities 
including bash, ssh, git, apt and many more. 
More 
Free 
Get 
Add to cart 
O Wish list 
Overview 
System Requirements 
Reviews 
Related

Terminal Commands

Once the above tools are installed, I opened Windows Terminal but you can use PowerShell or your command prompt. The next steps should be the same on each.

Once your tool of choice opens, the first command you want to enter is Ubuntu to get you into the environment.

You’ll need to install curl in order to perform the upcoming nvm install.

sudo apt install curl

Next, we need to install NVM with the following command.

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.36.0/install.sh | bash

Once this runs, you’ll want to restart your terminal; otherwise, you’ll see a message telling you that NVM wasn’t found when you try to use the nvm command.

Using NVM

Once I had the above setup completed, I seemed to be good to go in terms of using NVM on my Windows machine. So the next thing I wanted to see, was could I have 2 versions of Node v14.4.0 with different modules.

When you install a node version (v14.4.0 in my case), you simply use the following command:

nvm install v14.4.0

If you try to create an Alias, NVM will create a file in an alias directory with the name of the alias and inside the file is just a version number. It’s basically mapping your alias name to the version folder.

If you want to see where those files are, you can type “explorer.exe .” in your command line (note the dot after exe) and Windows Explorer will open in the directory that nvm is installed.

If you go into the .nvm directory, you’ll see 2 folders that we care about right now.

  • versions
  • alias

Inside versions, you’ll see a node directory and inside that, you’ll see each version of node that you installed. Inside those directory, you’ll see the modules that you installed.

Inside the alias directory, you’ll see a file for each alias that you create.

The Hack

Once I install Node v14.4.0, I will have a directory called v14.4.0 under /versions/node. I then created my first alias using the following command which tells nvm that we want to refer to the v14.4.0 directory as Contoso:

nvm alias Contoso 14.4.0

Next, I renamed the v14.4.0 directory to v14.4.0 Contoso. Then you need to make a change to the Contoso alias file since you changed the directory. Go to the alias directory, open the Contoso file and you should see one line with your version number. In my case, “14.4.0.” I added Contoso to that which tells nvm that when I use the Contoso alias (which matches my file name), load up whatever is in a folder called “v14.4.0 Contoso.”

Now we need to do the same steps for Fabrikam. Repeat the above steps which are:

  1. Install the node version so that the folder is created
nvm install v14.4.0

2. Create your alias

nvm alias Fabrikam 14.4.0

3. Rename the “v14.4.0” folder found at /.nvm/versions/node/ to “v14.4.0 Fabrikam”.

4. Go to /.nvm/alias/ and open the new Fabrikam file. Edit the single line inside the file to include Fabrikam at the end so that it matches the name in step 3.

Your renamed folders should look like the following:

Your alias folder should look like this:

The Test Drive

Before we install any modules, let’s take a look at each alias and what they have inside. As you can see, we can check each alias to see what modules we have and all we have is npm@6.14.5 in both to start.

I’m going to switch back to Contoso and install gulp.

I then switch to Fabrikam and install grunt.

So now if I go back to use Contoso and list out it’s modules, we’ll see NPM and Gulp.

Switching to Fabrikam, I can see the list of modules includes NPM and Grunt.

Issues

My Linux skills aren’t too hot these days so I did have some issues. I installed Yeoman and when trying to run the Yo command, I would get an EACCES error which meant that I didn’t have the right permissions to a particular config folder. I’m using the root account but had to run the following command to get past the issue:

chmod g+rwx /root /root/.config /root/.config/configstore

Basically we changed the permissions in that configstore folder to allow read, write, and execute for the user. I also had permissions issues in the npm folder and the folder where I was using yeoman so I ran the following commands to get past those.

sudo chown -R 1000:0 "/root/.npm"
chmod g+rwx /root .

The 1st line was a fix copied directly from error messages produced by the yeoman generator. The 2nd line was to remove a permissions issue thrown by yeoman in the folder where it was trying to create the project. I’m probably not following security best practices here, but these steps got me up and running.

Conclusion

In the end, I was able to get two instances of a node version using a different set of modules. If you are on Windows and can isolate by using different minor versions or if you don’t have an issue with having all your modules installed together, then NVM for Windows is all you will probably need. If you have a need to use the same version of Node for different projects that need to be separate, then the few steps above should get you there and using Windows Subsystem for Linux will allow you to install the original NVM so that you can leverage it’s alias feature.

M365 Philly Virtual Post Event Material

I wanted to provide a follow up to M365 Philly Virtual. At the time of the event, I wasn’t sure how we were going to share the recordings. A YouTube channel was created for SPS Philly which is the group that organizes the M365 Philly (formerly SharePoint Saturday Philly) as well as the Tri-State Office 365 User Group. The user group meets on the 2nd Tuesday of each month and will meet virtually for at least the remainder of 2020 and likely the first 3-6 months of 2021.

The recorded sessions take some time to edit so it’ll take some time before they’re all released but subscribe to the channel and get notified as the sessions are released.

I’m kicking around some ideas for content for the YouTube channel. I would prefer to release content occasionally on the channel so that it’s not only populated when we have an SPS event.

Want Content?

If you’re interested in following along, there are several places where you can do so.

For M365 Philly content, follow us on:

For Tri-State Office 365 User Group content, follow us on:

  • Twitter @TriStateO365
  • Meetup
  • If any sessions are recorded, they will be posted on the YouTube channel above.

Using AI Builder’s Form Processing Model and Flow to Extract Data from PDF Files

In the previous post, Getting Started with AI Builder’s Form Processing, I walked you through how to setup a model that will identify fields and their values in a pdf. In this post, I will show you how to use that processor to extract those values and store them as metadata for the pdf when it’s uploaded to a SharePoint document library – making the files filterable. Let’s dive right in.

Flows for AI Builder are special and do not follow the typical flow creation process. Typically, you would go to My Flows and start creating an automated flow. In this case, we want to go to Solutions and then click the New Solution button.

I’ll call this one SPL Invoice Processor. It also asks you to select a Publisher and I’m going to select the default publisher for the org. Your options are that or a CDS Default Publisher.

Once the solution is created, you’ll see it at the top of the page. Click on the link and then you can start creating your flow or other solutions. Think of SPL Invoice Processor as a container.

At the top of the page, you’re going to click on New and select Flow.

Give your Flow a name. Just to differentiate it a little, I’m going to call this flow SPL Invoice Processing Flow.

Next, you can select your trigger which in our case is going to be the “When a file is created in a folder” SharePoint trigger. In the screenshot, you can see that I’m searching for “file created” to narrow down my options below.

Once the correct trigger is selected, you can hit the create button and on the next screen, you’ll provide the url and document library name and then click the Next Step button to select your next action.

Next, you’ll want to filter down to the AI Builder actions and select Predict from the available actions.

Predict provides a single dropdown with a list of the available models. Select your model from the dropdown but if it’s not there, you may need to go back and publish your model. If it’s published, select “Enter custom value” which will turn the dropdown box into a text field where you can enter the name of your model. In my case, my model is called SPL Processor and it was in my dropdown. If you haven’t created a form processor at this point, check out my previous post where I walk through the steps needed to create one that will extract data from invoices. When you select a model, it’ll then ask you to select a document type and provide the document. Since I’m expecting pdf files, I’m going to enter “application/pdf” and provide the File Content that comes from the trigger.

We want to update the item fields values for the newly created/uploaded document. So I’m going to get the list of available properties first before I can update them. Add the Get File Metadata using Path action (If you use Get File Metadata, you’ll get an error when you try running the flow). This will allow you to get properties like the item id which will be needed to update that appropriate item in the library. For the File Identifier, we’re going to select x-ms-file-path-encoded which comes from the trigger.

Next, we are going to add the “Update file properties” action and get the details for our current file. We’re getting the item id from the previous action.

The Invoice Number, Client Name, Client Company, and Service Offered will come from the Predict actions output.

The end result will look like the following. Note that each entry that comes from Predict is the value so I’m providing “Invoice Number Value”, “Client Name Value”, and so on.

Now, it’s time to test this flow out. I’ll drop 5 invoices into the Invoices library and the following screenshot are my results.

One thing that you might notice above is that my invoices were 3 digits and you’re seeing a single digit invoice number above. That’s just SharePoint stripping the leading zero’s from the front of an integer. If I wanted to see 001 – 005, I could convert Invoice Number field into a Single Line of Text field. So there you have it. It’s pretty simple to automate the extraction of data from pdf files using AI Builder and Power Automate. This example was just a simple data extraction solution but you can do so much more. One thought that comes to mind is pulling the invoice total and if it’s above a certain number, kick off an approval process or notification. You can do the same if the number is too low. Maybe you have different steps depending on the type of service you’re offering? Once you can extract what you need, you can build all kinds of solutions based on that data. I encourage you to sign up for a free trial. It’s a pretty cool service.

Getting Started with AI Builder’s Form Processing

I’ve been speaking at conferences on various artificial intelligences topics lately and one of my demo’s is of AI Builder’s form processing. If you look around, you can find some examples of this and the common one is around extracting information from invoices. Invoices are the easiest types of forms to get a hold of so that’s probably why they all seem to be the same concept but with AI Builder’s form processing, the only requirement is that the forms are standardized.

Creating a Model

Getting started is super simple. AI Builder is an additional cost, but fortunately, you can kick the tires with a free trial. https://powerapps.microsoft.com/en-us/ai-builder/

AI Builder is a component of Power Platform so you can access it via Power Apps or Power Automate. In the next screenshot, you’ll see me accessing it from Power Automate. Under the AI Builder menu, you’ll want to go to the Build menu item which will show you the list of trainable models. (Lower on the page is a list of models that are already trained for you but we’ll be focusing on the Form Processing trainable model in this post).

AI Builder does a fantastic job of walking you through what you need to do. When you click on Form Processing, it’ll prompt you for a name for your AI model, it’ll tell you what you’ll need which in this case is 5+ documents with the same layout, and it’ll show you examples and best practices.

I’m going to name my model “SPL Processor” and hit the create button. Next, it’ll walk you through each step starting with uploading the 5+ documents that you will use to train the form processor.

I clicked the Add Documents button and it gives me some options for gathering those documents. I’m going to choose to Upload from local storage and I’ll upload 5 completed invoices from my local files.

A popup will appear showing you your selections and then you just need to click the Upload documents button.

It’ll then show you if the files uploaded successfully.

Analyzing Documents

After the files are uploaded and you hit the close button, you’ll be taken back to the Add Documents step and you’ll see your uploaded files plus an Analyze button below.

Once you hit the analyze button, a pop up will show you that it’s analyzing.

Identifying Fields that you want to Extract

Next, you’ll need to identify the spots in your files where you want to extract data. You’re documents should be a specific template so the fields should appear in the same spot each time.

Click the image of the file, and a pop up will again walk you through what it expects you to do. The processor seems to make an attempt of highlighting where the content will come from but this is your opportunity to define exactly what you want to extract. This is the processor’s attempt.

You can start highlighting the values that you want to pull out by dragging a box around the values. In the following image, you can see that I highlighted the invoice number and the callout shows and that is where you can provide the name of the field which I named “Invoice Number.” I’m going to repeat this process for the remainder of the fields.

The end result will look similar to the next image. There’s going to be a Confirm Fields button at the top right. Click on that button when your done identifying each of the required fields then repeat those steps for the other 4 documents that you uploaded.

The remaining documents are easier to tag. You can highlight the value and select the field from a list.

Training

Next, you’ll want to Train the model. You see a summary of your model, which includes the fields that you selected. It’s just informational so you can click the Train button here.

Now you’ll wait a few minutes as your model is training.

Once it’s trained, you can click on the model and you can perform some quick tests and if all looks good, go ahead and publish.

Taking it out for a Test Run

If you click the Quick Test button, you’ll get a popup where you can drag and drop a sample into. It’ll give you a visual of all of the fields that it found. when you hover over the fields, you’ll see the name of the field that it identified, the value, and the confidence score. I find that when you are working with a structured file with typed responses, the confidence score is usually 100%. In my example, the city, state, and zip are very close to each other so they may cross over their boundaries and in those cases, you’ll see slightly less confidence scores and by slightly less, I’ve seen 98%. I’ve seen the confidence score go dramatically lower on a handwritten form where the handwriting was sloppy and crossed boundaries. For example, when the client name crosses over where the processor is expecting to see the company name.

Conclusion

At this point, you now have an invoice processor that is trained against a specific template. Your processor doesn’t need to be an invoice. As long as you have a consistent template, you can use that. In my next post, I’ll show you how to create a Flow that will use the processor to extract values that will be saved to a SharePoint list.

M365 Philly Virtual 2020: Behind the Scenes

If you follow me, you may recall that last year, SharePoint Saturday Philly returned for the first time in 4 years with a new group organizing it. It was something that we had been talking about for a while at user group meetings and finally, we pulled the trigger and it was successful. If you attended, you probably saw me running from room to room, non-stop, constantly checking on speakers and sponsors with a piece of paper where I was jotting down any issues that came up. That became my lessons learned sheet and I was going to make a few improvements to the process and event this year.

The pandemic hits us and all public events at our venue get canceled for a month – at first. I host the Tri-State Office 365 User Group and we meet every month on the 2nd Tuesday so that cancelation forced us to cancel our March meeting. Then we got word that the venue was going to continue to be closed so we decided to go virtual in April. SPS (or M365 Saturday Philly as we were calling it) was scheduled for May 9th but it was becoming clear that it wasn’t going to happen. The user group meeting in April went well and we postponed the conference but we started kicking around the idea of going virtual. We weren’t sure what the logistics would look like, but then the Microsoft 365 Virtual Marathon happened and that was all we needed.

The Planning

We watched the marathon and some of the team actually participated in it. They had some technical issues at the beginning of the event but it stabilized throughout. I learned the logistics that went on with it and Mike (one of our organizers) and I spun up our own practice Live Event to see how the controls would work. Manpreet, another one of our organizers, also had a massive event scheduled for a week before M365 Philly so we used what we learned there to prepare for this event but before that event, we still had quite a bit of work to do.

Sometime during the planning phase, Tom reached out and offered to help. He got us connected with someone who was also involved in the M365 Virtual Marathon and we all got on a call to understand what we needed to do, things we needed to consider, issues that we might have, etc. That meeting was invaluable because as I said, Manpreet had a massive event coming up and the tips we got from that call potentially saved us from a lot of pain.

To Find Sponsors or Not to Find Sponsors, that was the Question

We’re nearing the event and we were behind in our marketing. I also went back and forth on whether or not to look for sponsors. The reason we get sponsors is to cover our expenses which include food for the attendees and any swag that you might give out. Since we were going virtual, my only expense was going to be the raffle so I didn’t put anywhere near the amount of effort that I did the previous year in looking for sponsors. I took the sponsor package that I created the year prior, removed any details that came with being onsite, and slashed the prices. Any money that came in would go to funding the next in-person event since I didn’t really have any expenses this year outside of the raffle. Technically, there was one more expense. I bought a cooler for the drinks prior to the cancelation of the in-person event. I did send emails out to a few businesses and that was near the date of the event so I wasn’t expecting to get sponsors. Typically, you want to start reaching out to businesses months in advance because they budget for things like that and reaching out sooner would allow them to account for it. The planning for this event was done in roughly 2 months which is shorter than the time you spend on an in-person event. In the end, businesses affiliated with the team sponsored the event.

Speakers

Mixed into all of the above was the reopening of the call to speakers. We had some submissions from the in-person event still in our queue but when we announced that it was going to be virtual we got an uptick in submissions from an international pool of speakers – which, thinking back on that was so cool. We ended up with 100+ sessions submitted. We used Sessionize to get session submissions but hadn’t used it before so we made some assumptions and I found the lack of documentation to be annoying. It’s not difficult but there are a few buttons where you’re thinking to yourself, “what’s about to happen here?”

What I did next was add the team to Sessionize to make the selections. I also configured it so that the speakers were anonymized (very cool feature by the way). When we were making the selections, the names were replaced with fake name. (Manpreet was wondering why he didn’t recognize any names for a day or two). Sessionize then puts 3 sessions side by side and you can rank them by 1st, 2nd, or 3rd. You can also skip or ignore a session. Let’s say you don’t know enough about the topic to make a decision, you can just ignore it. I don’t know what that does to the calculations though – like i said, the documentation is non-existent.

You also have the option of grouping the selections by their category so if the speaker says their session is a developer topic, we can pit the developer topics against each other or we can mix all the topics together and do it that way. The sessions also appear randomly so you will end up voting on a session multiple times but you would be ranking them against a different group of sessions.

The Selections

We did a lot of work in a short period of time. We put this event together in a shorter period of time than we would’ve if it were an in-person event. Because of that, I made the decision to keep the call for speakers open later than I should’ve. I should’ve closed it 1 or 2 weeks sooner to give the speakers time to prepare or make arrangements. I didn’t account for the number of submissions to be so much higher so I could’ve closed it sooner.

Once we ranked everyone’s sessions, Sessionize gives you an overall rank for each one and what we did next was we sorted each session from highest to lowest. We then just selected everyone from top to bottom and we skipped sessions where the speaker already had a session selected. Many speakers submitted 2 or more topics so there were a few sessions skipped. I also, initially, skipped my own session which was in the top 20 selections. (I wonder if the team knew that it was my topic). I ignored my own session when I saw it in the head to head rankings.

When the initial 33 accepted sessions went out, many of the speakers accepted. A few pulled their sessions because they weren’t available during the event and one didn’t confirm so we simply selected the next best presentation where the speaker had not already been selected. There was a point near the day of the event where we had 2 gaps to fill and I felt like that was too short of a notice to accept new speakers. Rather than asking the speakers to present a 2nd topic, Manpreet and I added our sessions to the schedule.

My original plan was to roam the event checking in on sessions. In the end, I produced the Eagles virtual room from beginning to end and placed my session on the first day to get it out of the way early so that I could focus on the event. Manpreet became the roamer, looking into issues as they came up.

The final 3 day lineup was impressive. Our keynote speakers were great, our speakers were informative, and the feedback was overwhelmingly positive.

Collateral

The cool promos, speaker banners, day schedules, and social media push were all Tom. He did a great job putting that material together. The tweets alone would’ve been very difficult to do while also focusing on the event. He made sure that something was going out as speakers were about to go live.

Producers

We had 9 producers (including the 4 organizers) to manage this event. You really need one producer per track and an additional person who can help with any technical issues that may come up. In our case we needed 4 producers on the first two days and only three for the last day plus that roamer/support person. The event ran 4 hours each day plus an additional hour on that last day for the wrap up. M365 Philly Virtual was in the middle of the work week, so we wanted more producers just to spread the load. I couldn’t ask the team to take 13 hours out of their work week. We had several people volunteer to help and we were able to give people a break. The team stayed on anyway, but the break allowed them to focus on support.

The Event Itself

For the most part, the event went well. He had the hiccup right out of the gate with the 1st keynote audio. I talk about it in a previous post but I neglected to enable system audio so the attendees couldn’t hear our audio. After that, day one went smoothly. The producers were able to handle any issues that speakers had and the feedback from day one was great.

Day 2 apparently was without incident. I am not aware of any issues at all. Day 3 however, had some issues. Apparently, there was a service degradation issue that day so our attendees were having issues watching the keynote and a little bit of the first sessions. It wasn’t everyone either. Two of our organizers were seeing issues on devices where they were logged in as attendees while another organizer and myself were not experiencing issues. Manpreet put in a call with Microsoft and got a quick response but there wasn’t anything we could do. I believe the suggestion was to log in with a mobile device, but one attendee reported in the Q&A that it didn’t matter which device he/she was using, the experience was the same. Shortly after the first speaker session began, all indications were that things started to stabilize.

My Day 3 Keynote Fumbles

While scrambling to figure out why we were having connection struggles, I became part of the show… and I did not see that coming. Going in, I knew that our 3rd keynote speaker likes to be interactive but Live Events doesn’t allow for much interactivity. The audience can ask questions via the chat, producers and speakers can see the questions but the audience can’t unless we publish them. There’s also a 10-15 second delay for the audience so if you ask them a question, you had a 10-15 second wait before they even began responding to it which is not idea for live interaction. While he was speaking, I was chatting with the team trying to figure out what the deal was with the connectivity issues. That’s when things got interesting.

So our keynote speaker asks a question that I partially heard. Something about musical collaboration. I scrambled and responded with “uh, I’m not good with pop culture references” but I should’ve guessed the answer because I’ve heard him ask that question before. Face palm! Apparently the question was what bands collaborated on “Walk this Way.”

So I hang my head and go back to the issue at hand and more questions started coming. I remember saying in the team chat “WHAT IS GOING ON?!” followed by the following gif.

That’s when Tom said “You’re part of the keynote now.” At that point, I left the issues to the team and I stayed focused on the talk. Now, I actually am bad at remembering pop culture stuff, and I probably would’ve only gotten half the question right was I not distracted. It was a funny experience thinking back on it. My wife gave me a hard time about that one too.

While I’m at it, there was another funny keynote moment that happened on day 2. I have a tendency to speak low in general but even more so when I speak on the phone or on a microphone. I don’t know why, I guess I feel like I’m yelling when I speak normally on those devices. Anyway, on day one, I was doing that very thing and I guess I came off as slightly monotone. Right before going live on day 2, Tom and I were having a conversation with Heather Newman and 1 minute before going live, Tom says “Hey Jason, use your excited voice” and goes on mute. My response was “wait, what?!” And then we go live.

Wrap it up

Overall, the event was simultaneously fun and stressful. We were doing something new – connecting an international group of speakers to an international audience over a virtual conference. We provided great value to the attendees. We connected our speakers to a new audience. We had people connecting from the US, UK, Netherlands, India, Mexico, Spain, Canada, Germany, China, and more. The 4 of us enjoyed providing this type of value to people and have done several events together. I’m grateful to have had the opportunity to work with this team in putting this event together and I look forward to working with them on future events.