Microsoft Sentinel — Upload Custom Log Files

4 min readJun 29, 2022

This post will describe how to ingest custom log files into a Log Analytics workspace using GitHub Actions!

Why yet another method

Since I started with Microsoft Sentinel in the days it was still called Azure Sentinel I found it difficult to send my own logging custom logging data to the attached Log Analytics workspace.

Although the Microsoft Sentinel GitHub page provides a good description of how to export data from a workspace, it doesn’t describe anything about uploading that same data to another workspace.

Somewhere in 2019 Microsoft docs were updated including a PowerShell script to upload data to a log analytics workspace. Although this solution works and is still often used in many Microsoft Sentinel Data Connectors for 3-party connectors, it still requires some coding skills.

The thing is this old script is not written very modular, and unfortunately is still copy/pasted into new data connectors and solutions causing a lot of legacy code duplication.

Recently Microsoft made the Azure Monitor HTTP Data Collector API in public preview.

Whoehoe! you might think, but unfortunately, the script hasn’t been changed and is still the same as already described in 2019! In this new era I would at least expect that:

To overcome this and make the script more readable, I’ve created an updated version that is available on my own GitHub

GitHub Actions rules the world!

Recently I started to play around with the SuperPowers of GitHub Actions are little building blocks that can be used in a yml pipeline, and are actually little wrappers around code.

The GitHub Actions uses code packages in Docker containers, which run on the GitHub servers and which, in turn, are compatible with any programming language.

Adding the Action

Open your GitHub repository and select Actions in the top ribbon menu Once the action page has opened select setup a workflow for yourself
Add the following code block to your workflow.

name: CustomLogs
on:
  push:
    paths:
      - samples/**jobs:
  custom-logs:
    name: Custom Logs
    runs-on: ubuntu-latest
    steps:
      - name: Check out repository code
        uses: actions/checkout@v3
      - name: Microsoft Sentinel Custom Logs
        uses: SecureHats/custom-logs@v1.0
        with:
          filesPath: samples
          workspaceId: ${{ secrets.WORKSPACEID }}
          workspaceKey: ${{ secrets.WORKSPACEKEY }}

By default, all files in the selected folder are processed. If you only want to process newly added files, an extra step needs to be added to the pipeline before the custom-logsaction.

- name: Get changed files
  id: changed-files
  uses: tj-actions/changed-files@v23.1
  with:
    separator: ","

This will output a comma-separated list that can be used as an input for the custom-logs action.

filesPath: '${{ steps.changed-files.outputs.all_changed_files }}'

The GitHub Action uses a couple of parameters to control to which workspace the logs files need to be sent and what folder is processed.

In the table below you can find a description of these parameters.

3. As you might have noticed 2 secrets are used in this action. These need to be created first before we can run the workflow.

These secrets can be created by following this documentation link.

Testing the Action

After everything has been set up correctly we can start testing out GitHub Action! Depending on how the workflow in step #2 has been configured, the Action is triggered on a push to a specific folder. In this case, we configured the samples folder as the trigger folder on line 5 and line 17 as the input folder for the action.

So the only this we now have to do is drop a file in the samples folder. As soon as the file has been pushed to the folder, the GitHub Action starts to run as shown in the image