Azure Stream Analytics - Troubleshooting Issues

Stream Analytics monitoring and logs

There are several ways how to monitor and track down loads and all other sort of issues that might occur in processing Azure Stream Analytics:

1. Activity Log
2. Diagnose and solve problems
3. Metrics
4. Alert Rules
5. Diagnostic Logs

My favorite and the most effective way to troubleshoot event issues in the most of cases is definitely Activity Log, which will be our focus here as well as Metrics graphical tool and statistics.

Also, there are 2 other useful tools in "Support + Troubleshooting Section" of Stream Analytics Jobs:

1. Resource Health
2. Job Diagram - check data driven debugging with Job Diagram

Other Resources

For more information on troubleshooting refer to: Troubleshooting resources on Microsoft Docs

Troubleshooting Time Management Issues

Because we are dealing with live streams and real time loads, there are scenarios where events are not syncing in desired way and some of the events sent to Real Stream Analytics may be delayed and dropped. This might be caused either by application times or early arrivals of the events between client application and Stream Analytics job.
  
For broader consideration, please refer to and explore how to detect and resolve latency issues:

• Time management
• Time Skew Policies
• Late/early arrivals

Activity Log - Troubleshooting Demo

Here is just a quick demo on how to monitor and track down event issues.

1. Activity Log - Navigate to Stream Analytics job and click on "Activity Log" tab





2. Explore filters and log history







3. Activity Log filters - the entire error log might be overwhelming and let's narrow down results to 24 hours in this case






4. Error log - filter down "Event Severity" to errors only and now we have the clearer picture on what is going on






5. Review individual error logs - usually, summary section doesn't show enough of information or there is no error description at all, but JSON normally reveals the right information






6. JSON log output - in this case, there is an an error of mismatching source and destination column types (System.Double vs System.Int32). This looks like a miss in proper mapping between input and output in the input and output of the Stream Analytics job and can be resolved by fixing either of the data types. Also, for this demonstration, only a snippet of JSON was shown below, but browsing to the entire JSON document can show detailed path to both, inputs and output as well as other information related to the resources used and error message produced.






7. More troubleshooting - After fixing the previous error message, there was still one more left and it's about missing column in the destination. It seems that the destination column wasn't named properly and renaming column to match the same name in input resolves the issue.

Metrics

Metrics section might be very useful in certain cases like troubleshooting for  early/late arrivals, data conversion errors, input/output events, runtime errors...