Struggling to troubleshoot your code in Snowflake? You need to read this.

Thank you for reading my latest article Struggling to troubleshoot your code in Snowflake? You need to read this..

Here at LinkedIn I regularly write about modern data platforms and technology trends. To read my future articles simply join my network here or click ‘Follow’. Also feel free to connect with me via YouTube.

Introduction to Event Tables in Snowflake

For data engineers working in Snowflake, event tables are a powerful tool to capture and analyze the behavior of your Snowflake function and procedure handler code, including code developed using Snowpark APIs. Event tables allow you to collect log messages and trace events in real-time as your code executes, enabling detailed monitoring and diagnostics.

In this blog, we’ll keep it simple and focus on what event tables are, how they work, and how you can use them effectively.

What is an Event Table?

An event table is a special type of database table in Snowflake that automatically logs telemetry data, which includes information about the execution of Snowflake objects (like procedures, UDFs, and functions). These tables follow a predefined structure that supports the OpenTelemetry data model, a widely-used standard for telemetry data.

To enable storage of telemetry data, you must specify that an event table is an active event table.

After installation, Snowflake includes a default event table called SNOWFLAKE.TELEMETRY.EVENTS. This event table is active and collects data until you deactivate it. You can also create your own. To create and activate your own event table for your account, execute the ALTER ACCOUNT command, and set the EVENT_TABLE parameter to the name of your event table.

ALTER ACCOUNT SET EVENT_TABLE = my_database.my_schema.my_events;

What Does an Event Table Capture?

• Log Messages: These are detailed messages that provide independent information about the state of a specific piece of your code. Log messages are crucial for debugging and understanding the flow of execution.
• Trace Events: These events provide structured data that span multiple parts of your code. They allow you to group related activities, making it easier to trace the flow of execution through different Snowflake objects.

How to Use Event Tables

1. Activate an Event Table: Create and activate your own custom event tables. The event table becomes the destination for all telemetry data.
2. Collect Telemetry Data: To collect telemetry data, ensure that your event table is active and that the telemetry settings are properly configured. This setup allows Snowflake to capture the events emitted by your code.
3. Query Event Data: Once your telemetry data is collected, you can query the event table using simple SQL queries. This allows you to analyze log messages and trace events to gain insights into how your code is behaving in real-time.

Costs of Telemetry Data Collection

Telemetry data collection incurs storage costs. To manage costs, you can:

• Limit the amount of data you collect by adjusting telemetry levels.
• Regularly clean up old data in your event tables.
• Use filtering to capture only the most relevant events.

For more details, refer to Snowflake’s documentation on managing telemetry costs.

Summary

Event tables in Snowflake provide a straightforward way to monitor and analyze your code execution. By capturing log messages and trace events, you can gain deep insights into your Snowflake procedures, functions, and Snowpark code, ensuring you can troubleshoot and optimize as needed.

To stay up to date with the latest business and tech trends in data and analytics, make sure to subscribe to my newsletter, follow me on LinkedIn, and YouTube, and, if you’re interested in taking a deeper dive into Snowflake check out my books ‘Mastering Snowflake Solutions’ and ‘SnowPro Core Certification Study Guide’.

About Adam Morton

Adam Morton is an experienced data leader and author in the field of data and analytics with a passion for delivering tangible business value. Over the past two decades Adam has accumulated a wealth of valuable, real-world experiences designing and implementing enterprise-wide data strategies, advanced data and analytics solutions as well as building high-performing data teams across the UK, Europe, and Australia.

Adam’s continued commitment to the data and analytics community has seen him formally recognised as an international leader in his field when he was awarded a Global Talent Visa by the Australian Government in 2019.

Today, Adam is dedicated to helping his clients to overcome challenges with data while extracting the most value from their data and analytics implementations. You can find out more information by visiting his website here.

He has also developed a signature training program that includes an intensive online curriculum, weekly live consulting Q&A calls with Adam, and an exclusive mastermind of supportive data and analytics professionals helping you to become an expert in Snowflake. If you’re interested in finding out more, check out the latest Mastering Snowflake details.