Chapters: 

We have an existing SPO metrics list, populated automatically by user activity, and we want to produce a straightforward report from that data.

No new data collection, no migration, no unnecessary infrastructure. And no unnecessary charges!

Reporting options

Option

What it provides

Complexity

1. SharePoint list views

Filter, sort, group and total metrics directly in SPO

Very low

2. Excel connected to SPO

Tables, pivot tables, charts and summaries

Low

3. Power BI

Interactive reports, charts, filtering and aggregation

Medium

4. Power Apps

A custom, admin-only interface for viewing metrics

Medium

5. Python-generated HTML

A standalone report generated from SPO data

Low, given your existing tools

6. Dataverse

A separate data platform that can expose or store SPO metrics

Higher

What each option actually means

1. SharePoint views

We can create views of the existing metrics list, including grouping, filtering and totals for numeric columns.

No additional platform or data duplication.

The limitation is presentation. It's still a SharePoint list view, not a particularly attractive report.

2. Excel

Excel can connect to a SharePoint Online list using Power Query.

Once connected, you can create pivot tables, charts and summaries. Refresh the connection when you need updated metrics.

This is probably the simplest route to a conventional management report.

3. Power BI

Power BI Desktop can connect directly to your SPO list.

You can create reports showing, for example, searches by day, most frequently accessed pages and activity trends.

You don't need Dataverse or Power Automate.

Power BI Desktop is free for authoring. Publishing and sharing reports through the Power BI service may require additional licensing.

4. Power Apps

A canvas app can connect directly to the existing SPO list.

You could create an admin screen displaying totals, filtered records and basic charts.

However, Power Apps is primarily an application-building platform. For reporting alone, it's probably more work than necessary.

5. Python and HTML

This is close to what you're already doing.

Python reads the metrics from SharePoint, processes them and generates HTML.

Advantages include complete control over formatting, no additional reporting platform, and the ability to run the report manually.

The disadvantage is maintaining your own reporting code.

6. Dataverse

Dataverse is worth understanding, but it isn't necessary to produce this report.

A SharePoint virtual table can expose list data through Dataverse without copying it. However, virtual tables have limitations around aggregation and native Dataverse reporting features.

Alternatively, you could replicate the metrics into regular Dataverse tables, but that introduces synchronization, storage and security considerations.

For a simple report, neither approach offers an obvious advantage.

One important distinction: reporting versus collection

                   USER ACTIVITY
                         |
                         v
                  SPO Metrics List
                         |
            +------------+------------+
            |            |            |
            v            v            v
       Excel / BI    SharePoint     Python
                       Views         HTML

All three reporting approaches can read the same existing list.

The metrics continue accumulating regardless of which reporting tool you choose or how often you run a report.

 

Yes. Based on what you've already established:

  • Data source: Existing SharePoint Online metrics list.

  • Purpose: Evaluate how users find and use Plays in the Playbook.

  • Metrics: Searches, clicks, repeat searches, unique users (anonymous), and most-used Plays.

  • Existing solution: Python-generated HTML reports, run manually.

  • Management response: Positive, particularly regarding presentation.

  • Requirement: Professional-looking, browser-accessible reporting without unnecessary complexity.

Evaluation

Option

Fit

Assessment

Python HTML

Excellent

Already working, proven presentation, no new platform.

Power BI

Excellent

Best alternative for professional dashboards, directly connects to SPO.

Excel

Good

Can analyze all these metrics, but less suited to browser-first presentation.

SharePoint views

Fair

Useful for basic counts and grouping, but limited presentation.

Power Apps

Fair

Can create a polished interface, but requires building and maintaining an app.

Dataverse

Poor

Adds infrastructure without solving a reporting problem.

Conclusion

Power BI is the only alternative I'd seriously evaluate against your existing Python HTML reports.

It can use the existing SPO list, present all five metrics professionally, and support interactive reporting without introducing Dataverse.

The deciding considerations are Power BI licensing, report-sharing permissions, and whether its built-in presentation offers anything valuable beyond your existing HTML.

There is no technical reason, based on the requirements you've described, to replace the Python solution.

There is a manual export step, and the SP REST API is really XML in REST clothing!

Fair criticism. SharePoint's REST API has a rather impressive ability to make modern technology feel like 2007. :)

But there's an important distinction: SharePoint REST can return JSON instead of Atom/XML. You don't need to parse XML in Python.

For example:

GET /_api/web/lists/GetByTitle('Metrics')/items
Accept: application/json;odata=nometadata

That header requests a simplified JSON response.

Your Python code could then work with ordinary dictionaries:

items = response.json()["value"]
No XML parsing, no SOAP, no additional licensing.

Microsoft documents JSON responses for SharePoint list items, so this is supported functionality rather than a workaround.

The real obstacle is authentication, not data format.

Before we write any code, I'd investigate whether your existing authenticated SharePoint session can be used for a read-only proof of concept.

If that works, we can concentrate on automating the export without changing anything else in your reporting system.

Authentication

Look at Microsoft Entra ID delegated authentication using my existing Microsoft 365 work account.

There are two distinct possibilities:

Method

How it works

Suitable for

Interactive OAuth 2.0 (MSAL)

I sign in through Microsoft using my work account and MFA

Running Python reports on demand

Application authentication (certificate)

A registered application authenticates without a person signing in

Fully unattended scheduled reports

Both use Microsoft's existing identity infrastructure. Neither inherently requires a premium Power Platform licence.

Microsoft Learn
+1

 

My preference for your situation

Start with interactive delegated authentication:

  1. Python initiates a Microsoft sign-in.

  2. You authenticate with your existing work account.

  3. Microsoft issues an access token.

  4. Python uses that token to read the metrics list.

  5. Your existing reporting code generates the HTML.

There is one catch: Python needs an approved application identity (client ID), and the tenant must permit the necessary delegated permissions. We can't assume your browser's existing SharePoint session is directly reusable.

But there's an important distinction

You originally asked about automating the manual export.

If you want to run the reports yourself with one command, interactive authentication is appropriate.

If you want the reports to generate automatically every night, we need unattended authentication, which generally means an Entra application registration and certificate.

Which level of automation are we pursuing: one-command execution or completely unattended scheduling?

That determines whether we can keep authentication interactive.


Hi Kaiyan,

I'm investigating automating the retrieval of metrics from an existing SharePoint Online list using Python, without additional licensing or services.

Could you confirm three things?

  1. Interactive authentication: Can we use an existing or new Entra ID app registration with MSAL and delegated read permissions to access SPO list data using our Microsoft 365 credentials?
  2. Unattended authentication: Would certificate-based application authentication be permitted if we later want to schedule the reports automatically?
  3. Licensing: Can we accomplish this entirely within our existing Microsoft 365 licensing, without additional costs?

The objective is simply to replace a manual list export:

SPO metrics list → Python → HTML reports

No additional platforms or infrastructure.


Dataverse

This is an interesting use case for Dataverse because the SharePoint Online (SPO) list is already collecting operational metrics, and we're exploring what Dataverse might add to the reporting side.

The first thing to understand is that Dataverse is not simply another way to view a SharePoint list. It's a separate data platform with its own tables, security, relationships, and licensing.

And that distinction matters quite a bit for your metrics.

1. Three ways to approach this

Approach

What happens

Keep data in SPO

Your existing metrics list remains the source of truth. Reports query SharePoint directly.

Copy data to Dataverse

Power Automate or another integration copies metrics into Dataverse tables.

Use a virtual table

Dataverse exposes the SharePoint data through a supported virtual-table connector, without storing another copy.

2. What changes when you introduce Dataverse?

Current architecture

User clicks / searches
       |
       v
SPO Metrics List
       |
       v
Existing admin report

Possible Dataverse experiment
SPO Metrics List
       |
       v
Dataverse Table
       |
       v
Power Apps / Power BI / Dataverse views
This keeps the production metrics collection untouched while you investigate Dataverse.
 

3. Important discovery: SharePoint virtual tables ARE supported!

Microsoft now documents a SharePoint virtual connector provider. That means you can expose an SPO list through Dataverse without copying its records.

This is worth exploring first.

There are limitations, particularly relevant to metrics:

  • Virtual tables don't support native Dataverse charts, rollups, or calculated columns.                                -- > OK

  • SharePoint Person and Managed Metadata columns aren't supported by this connector.                        -- > OK

  • Virtual-table queries have a 1,000-record return limit, so reporting over substantial event histories needs careful consideration.

      Microsoft Learn                     

     

  1. Five Things to Investigate

Area

Question

Why It Matters

Environment

Do we have access to a Power Platform environment with Dataverse?

Determines whether Dataverse is available for exploration and testing.

Permissions

Do we have permission to create Dataverse tables?

The Environment Maker role alone does not necessarily grant table-creation privileges.

Licensing

Does our licensing include Dataverse capabilities?

Full Dataverse functionality generally requires premium licensing beyond standard Microsoft 365.

Data Volume

How many clicks and searches are recorded daily?

Replicating every event could increase storage, processing and licensing costs.

Security

Can reporting access be restricted to administrators?

Dataverse security roles can control access to tables and reporting data.

Current assessment: The existing SharePoint metrics list and manually generated reports meet the immediate reporting requirement. Dataverse remains an option for future investigation rather than an immediate migration priority.

Microsoft documents these distinctions in its licensing overview and security-role guidance.

5. Where I'd begin exploring

Go to Power Apps and examine the environment selector.

Look for Tables, Solutions, and whether you have access to Dataverse. You don't need to change your production list to investigate.

One particularly useful experiment would be to expose a small copy of your metrics list through a SharePoint virtual table, then see what Dataverse can actually do with it.

I wouldn't introduce Power Automate or a second database yet.