KQL query map SQL query

To those whom are familiar SQL query syntax, but new to KQL. The following table shows sample queries in SQL and their KQL equivalents.

CategorySQL QueryKusto QueryLearn more
Select data from tableSELECT * FROM dependenciesdependenciesTabular expression statements
SELECT name, resultCode FROM dependenciesdependencies | project name, resultCodeproject
SELECT TOP 100 * FROM dependenciesdependencies | take 100take
Null evaluationSELECT * FROM dependencies
WHERE resultCode IS NOT NULL
dependencies
| where isnotnull(resultCode)
isnotnull()
Comparison operators (date)SELECT * FROM dependencies
WHERE timestamp > getdate()-1
dependencies
| where timestamp > ago(1d)
ago()
SELECT * FROM dependencies
WHERE timestamp BETWEEN ... AND ...
dependencies
| where timestamp between (datetime(2016-10-01) .. datetime(2016-11-01))
between
Comparison operators (string)SELECT * FROM dependencies
WHERE type = "Azure blob"
dependencies
| where type == "Azure blob"
Logical operators
-- substring
SELECT * FROM dependencies
WHERE type like "%blob%"
// substring
dependencies
| where type has "blob"
has
-- wildcard
SELECT * FROM dependencies
WHERE type like "Azure%"
// wildcard
dependencies
| where type startswith "Azure"
// or
dependencies
| where type matches regex "^Azure.*"
startswith
matches regex
Comparison (boolean)SELECT * FROM dependencies
WHERE !(success)
dependencies
| where success == False
Logical operators
Grouping, AggregationSELECT name, AVG(duration) FROM dependencies
GROUP BY name
dependencies
| summarize avg(duration) by name
summarize
avg()
DistinctSELECT DISTINCT name, type FROM dependenciesdependencies
| summarize by name, type
summarize
distinct
SELECT name, COUNT(DISTINCT type)
FROM dependencies
GROUP BY name
dependencies
| summarize by name, type | summarize count() by name
// or approximate for large sets
dependencies
| summarize dcount(type) by name
count()
dcount()
Column aliases, ExtendingSELECT operationName as Name, AVG(duration) as AvgD FROM dependencies
GROUP BY name
dependencies
| summarize AvgD = avg(duration) by Name=operationName
Alias statement
SELECT conference, CONCAT(sessionid, ' ' , session_title) AS session FROM ConferenceSessionsConferenceSessions
| extend session=strcat(sessionid, " ", session_title)
| project conference, session
strcat()
project
OrderingSELECT name, timestamp FROM dependencies
ORDER BY timestamp ASC
dependencies
| project name, timestamp
| sort by timestamp asc nulls last
sort
Top n by measureSELECT TOP 100 name, COUNT(*) as Count FROM dependencies
GROUP BY name
ORDER BY Count DESC
dependencies
| summarize Count = count() by name
| top 100 by Count desc
top
UnionSELECT * FROM dependencies
UNION
SELECT * FROM exceptions
union dependencies, exceptionsunion
SELECT * FROM dependencies
WHERE timestamp > ...
UNION
SELECT * FROM exceptions
WHERE timestamp > ...
dependencies
| where timestamp > ago(1d)
| union
(exceptions
| where timestamp > ago(1d))
JoinSELECT * FROM dependencies
LEFT OUTER JOIN exceptions
ON dependencies.operation_Id = exceptions.operation_Id
dependencies
| join kind = leftouter
(exceptions)
on $left.operation_Id == $right.operation_Id
join
Nested queries
Sub-query
SELECT * FROM dependencies
WHERE resultCode ==
(SELECT TOP 1 resultCode FROM dependencies
WHERE resultId = 7
ORDER BY timestamp DESC)
dependencies
| where resultCode == toscalar(
dependencies
| where resultId == 7
| top 1 by timestamp desc
| project resultCode)
toscalar
HavingSELECT COUNT(\*) FROM dependencies
GROUP BY name
HAVING COUNT(\*) > 3
dependencies
| summarize Count = count() by name
| where Count > 3
summarize
where

Kusto Query Language (KQL) – quick reference

It is for the new KQL engineer to quick reference only.

KQL Work Flow

Quick reference

Operator/FunctionDescriptionSyntax
Filter/Search/ConditionFind relevant data by filtering or searching
whereFilters on a specific predicateT | where Predicate
where contains/hasContains: Looks for any substring match
Has: Looks for a specific word (better performance)
T | where col1 contains/has "[search term]"
searchSearches all columns in the table for the value[TabularSource |] search [kind=CaseSensitivity] [in (TableSources)] SearchPredicate
takeReturns the specified number of records. Use to test a query
Notetake and limit are synonyms.
T | take NumberOfRows
caseAdds a condition statement, similar to if/then/elseif in other systems.case(predicate_1, then_1, predicate_2, then_2, predicate_3, then_3, else)
distinctProduces a table with the distinct combination of the provided columns of the input tabledistinct [ColumnName], [ColumnName]
Date/TimeOperations that use date and time functions
agoReturns the time offset relative to the time the query executes. For example, ago(1h) is one hour before the current clock’s reading.ago(a_timespan)
format_datetimeReturns data in various date formats.format_datetime(datetime , format)
binRounds all values in a timeframe and groups thembin(value,roundTo)
Create/Remove ColumnsAdd or remove columns in a table
printOutputs a single row with one or more scalar expressionsprint [ColumnName =] ScalarExpression [',' ...]
projectSelects the columns to include in the order specifiedT | project ColumnName [= Expression] [, ...]
Or
T | project [ColumnName | (ColumnName[,]) =] Expression [, ...]
project-awaySelects the columns to exclude from the outputT | project-away ColumnNameOrPattern [, ...]
project-keepSelects the columns to keep in the outputT | project-keep ColumnNameOrPattern [, ...]
project-renameRenames columns in the result outputT | project-rename new_column_name = column_name
project-reorderReorders columns in the result outputT | project-reorder Col2, Col1, Col* asc
extendCreates a calculated column and adds it to the result setT | extend [ColumnName | (ColumnName[, ...]) =] Expression [, ...]
Sort and Aggregate DatasetRestructure the data by sorting or grouping them in meaningful ways
sort operatorSort the rows of the input table by one or more columns in ascending or descending orderT | sort by expression1 [asc|desc], expression2 [asc|desc], …
topReturns the first N rows of the dataset when the dataset is sorted using byT | top numberOfRows by expression [asc|desc] [nulls first|last]
summarizeGroups the rows according to the by group columns, and calculates aggregations over each groupT | summarize [[Column =] Aggregation [, ...]] [by [Column =] GroupExpression [, ...]]
countCounts records in the input table (for example, T)
This operator is shorthand for summarize count()
T | count
joinMerges the rows of two tables to form a new table by matching values of the specified column(s) from each table. Supports a full range of join types: fullouterinnerinneruniqueleftantileftantisemileftouterleftsemirightantirightantisemirightouterrightsemiLeftTable | join [JoinParameters] ( RightTable ) on Attributes
unionTakes two or more tables and returns all their rows[T1] | union [T2], [T3], …
rangeGenerates a table with an arithmetic series of valuesrange columnName from start to stop step step
Format DataRestructure the data to output in a useful way
lookupExtends the columns of a fact table with values looked-up in a dimension tableT1 | lookup [kind = (leftouter|inner)] ( T2 ) on Attributes
mv-expandTurns dynamic arrays into rows (multi-value expansion)T | mv-expand Column
parseEvaluates a string expression and parses its value into one or more calculated columns. Use for structuring unstructured data.T | parse [kind=regex [flags=regex_flags] |simple|relaxed] Expression with * (StringConstant ColumnName [: ColumnType]) *...
make-seriesCreates series of specified aggregated values along a specified axisT | make-series [MakeSeriesParamters] [Column =] Aggregation [default = DefaultValue] [, ...] on AxisColumn from start to end step step [by [Column =] GroupExpression [, ...]]
letBinds a name to expressions that can refer to its bound value. Values can be lambda expressions to create query-defined functions as part of the query. Use let to create expressions over tables whose results look like a new table.let Name = ScalarExpression | TabularExpression | FunctionDefinitionExpression
GeneralMiscellaneous operations and function
invokeRuns the function on the table that it receives as input.T | invoke function([param1, param2])
evaluate pluginNameEvaluates query language extensions (plugins)[T |] evaluate [ evaluateParameters ] PluginName ( [PluginArg1 [, PluginArg2]... )
VisualizationOperations that display the data in a graphical format
renderRenders results as a graphical outputT | render Visualization [with (PropertyName = PropertyValue [, ...] )]

Create Service Principle, Register an application on Azure Entra ID (former Active Directory)

A Service Principal in Azure is an identity used by applications, services, or automated tools to access specific Azure resources. It’s tied to an Azure App Registration and is used for managing permissions and authentication.

The Microsoft identity platform performs identity and access management (IAM) only for registered applications. Whether it’s a client application like a ADF or Synapse, Wen Application or mobile app, or it’s a web API that backs a client app, registering establishes a trust relationship between your application and the identity provider, the Microsoft identity platform.

This article is talking on registering an application in the Microsoft Entra admin center. I outline the registration procedure step by step.

Summary steps:

  1. Navigate to Azure Entra ID (Azure Active Directory)
  2. Create an App Registration
  3. Generate Client Secret,
    note down Important the Application (client) ID and Directory (tenant) ID,  Client-Secret-value.
  4. Using the Service Principle – Assign Roles to the Service Principal
    Navigate to the Azure resource (e.g., Storage Account, Key Vault, SQL Database) you want your Service Principal to access.

Step by Step Demo

Register a new Application on Azure Entra ID (formerly called Azure Active Directory), get an Application ID and Client Secret value.

Azure Portal >> Azure Entra ID (formerly called Azure Active Directory) 

(1)  Copy Tenant ID.

We need this Tenant ID later.

(2) App Registration

(3) Copy Application ID. We will use it later

(4) Create Client Secret

Generate a new client Secret,

(5) copy the Client Secret Value

Copy client-secret-value, we need it later.

Cause: the Client Secret Value you HAVE TO COPY IT RIGHT NOW! IMMEDIATELY copy NOW. And put it to a secure place. Since the Value WILL NOT reappear anymore. IMOPRTANT!

(6) Using the Service Principle – Assign Roles to the Service Principal

Assign Roles to the Service Principal

Now, assign permissions to your Service Principal so it can access specific Azure resources:

  1. Navigate to the Azure resource (e.g., Storage Account, Key Vault, SQL Database) you want your Service Principal to access.
  2. Go to Access Control (IAM).
  3. Click Add and choose Add role assignment.
  4. Choose a role (e.g., Contributor, Reader, or a custom role).
  5. Search for your App Registration by its name and select it.
  6. Save

We have finished all at Azure Entra ID (Former Azure Active Directory) 

Please do not hesitate to contact me if you have any questions at william . chen @mainri.ca

(remove all space from the email account 😊)

Appendix: Microsoft: Register an application with the Microsoft identity platform