Post-Audit Data Reporting Review
Thomas and Sarah discuss the results of an internal audit that identified data conversion errors in Brankside Holdings reporting. They review the implemented fixes, including switching to Parquet format and adding automated validation, and plan next steps for improving data quality and compliance.
What you’ll be able to do
- By the end, you can identify specific data transformation issues flagged by an internal audit.
- By the end, you can explain how file format changes and validation tools improve data integrity.
- By the end, you can describe timestamp standardization and reporting enhancements for stakeholder visibility.
- By the end, you can outline the steps taken to formalize data processes and communicate changes to stakeholders.
Dialogue
Beginner version
Thomas
Good morning, Sarah. Did you read the audit report? It came last Friday.
Sarah
Yes, I read it. I read it on the weekend. The team found the problem. It was the CSV file.
Thomas
I thought so too. The numbers were wrong. The script used a bad number type. So the totals were not correct.
Sarah
Yes, that is right. Also, one column did not keep four decimal places. Our rules say it must. So the audit team saw a problem.
Thomas
I fixed it. Now we save the data in Parquet. Parquet is better than CSV. It keeps the data correct.
Sarah
Good idea. The audit team wants one more check. They want to check the numbers after we load the data. They want to see if the numbers match.
Thomas
I work on that now. I use a tool. It is called Pandera. It checks every column. It sends an alert if something is wrong.
Sarah
That is great. We must add this check to our process. We do it before we send the report. Did you also change how we set risk levels?
Thomas
Yes. The script now gets risk data from our risk tool. It puts each product in a group. The groups are Low, Medium, and High. Old products get the label Unknown for now.
Sarah
Good. That makes things simpler. The audit team had another problem too. Some times were local time. Some times were UTC. That was not the same.
Thomas
I saw that too. Now all times are UTC+0. I also give each item a unique number. That helps us find it later.
Sarah
That helps a lot. The team also said we must write down all changes. We must say why we made each change.
Thomas
I changed the report. Now it has a new part. That part shows every big change. It says what changed and when and why.
Sarah
Very good. One more thing. The compliance team wants a Power BI dashboard. It must show new data every day. It must also show warnings and other numbers.
Thomas
I started that already. I use Azure Synapse. The dashboard shows changes each day. It shows problems too. I think it is ready by end of this week.
Sarah
That is very helpful. Did you talk to the Data Governance team?
Thomas
Yes. I showed them a diagram. It shows each step in our data process. I also wrote notes about the steps.
Sarah
Well done. After we finish, we must test the system. We can put wrong data in. Then we see what happens.
Thomas
Good idea. I made a test space in Snowflake. It is like the real system. We can test there. It is safe.
Sarah
Great. We also need to write everything down. The audit team liked your word list. Can we add more? We can add steps for errors and backup plans.
Thomas
Yes. I work with the writer on our team. We make a big guide. It has simple steps and notes. We also save all versions in Git.
Sarah
After that, we must tell the front office. They need to know about the changes. The changes affect their work.
Thomas
I made some slides. They show how we clean data. They show how it helps with trading. They talk about risk too.
Sarah
That is just what we need. Thank you, Thomas. We had a small problem. Now we made big improvements.
Thomas
It was hard work. But I am happy. Good data is very important. It helps everyone trust the system.
Sarah
I agree. Let us meet again when the dashboard is ready. Then we write a short note for the whole data team.
Thomas
OK. I write a first version of the note today. I send it to you. You can check it.
Sarah
Perfect. Thank you, Thomas. Very good work.
Thomas
Thank you, Sarah. Talk to you soon.
Intermediate version
Thomas
Good morning, Sarah. Have you had time to look at the audit report from last Friday?
Sarah
Yes, I went through it over the weekend. The team traced the problem back to how the data was converted into CSV format.
Thomas
That's what I suspected too. The numbers were being rounded too much because the script was using the wrong number type, so the totals came out incorrect.
Sarah
Exactly. On top of that, the 'contract_value_eur' column wasn't keeping four decimal places, which is required by our reporting standards, so the audit team flagged it.
Thomas
I've fixed that. The data is now stored in Parquet format instead of CSV, because Parquet handles data types and structure much better.
Sarah
That's a smart move. The audit team also wants a second check on the numbers after loading. They want the system to compare the final figures with the original ones.
Thomas
I'm working on that now. I'm using a tool called Pandera, which checks every column and sends an alert if it finds something unusual. If a key value is too different, it will raise an error.
Sarah
That's great. We need to include that check in our quality process before we send the final report. Did you also update the way we assign risk levels?
Thomas
Yes. The updated script now pulls risk data directly from our risk tool and sorts each product into one of three groups: Low, Medium, or High. Older products will be marked as 'Unknown' unless we add the data by hand.
Sarah
That should make reporting more straightforward. The audit team also raised an issue with inconsistent timestamps, some entries were in local time and others in UTC.
Thomas
I noticed that too. I've set all timestamps to UTC+0, following the ISO 8601 standard. Each entry also now has a unique ID to make tracking easier.
Sarah
That will make future audits much easier. The team also made clear that our report needs to document all data changes and explain the reasons behind them.
Thomas
I've updated the report format to reflect that. There's now a dedicated section that lists every major change, like table joins or data additions, along with who made them and when.
Sarah
Excellent. One more thing: the compliance team has asked for a Power BI dashboard. It needs to show daily data updates, as well as extra metrics like warning counts and confidence levels.
Thomas
I've already started building it. I'm using Azure Synapse to bring the data streams together. The dashboard will show daily changes and highlight any outliers. I expect it to be ready by the end of this week.
Sarah
That will give everyone much better visibility. Have you had a chance to speak with the Data Governance team?
Thomas
Yes. I shared a flowchart that shows each stage of our data pipeline, along with descriptions of the processes and the data involved at each step.
Sarah
Very thorough. Once we've finished these updates, we should run a stress test. We can feed in incorrect data to see how the system handles it.
Thomas
Good thinking. I've set up a test environment in Snowflake that mirrors our live system, so we can run tests without any risk to production.
Sarah
Great. We also need to document everything properly. The audit team liked your glossary, can we expand it to cover error-handling steps and backup procedures?
Thomas
Absolutely. I'm working with our documentation specialist to put together a full guide with clear step-by-step instructions. We'll also track all versions using Git.
Sarah
Once that's done, we should update the front office. They need to know how these changes affect their day-to-day work.
Thomas
I've already put together a short slide deck. It shows how our data cleaning affects their trading decisions, especially around risk and capital allocation.
Sarah
That's exactly the kind of teamwork across departments that we need. Thank you, Thomas. We started with a small discrepancy and turned it into a real improvement.
Thomas
It's been a big learning process, but I'm really pleased with how the team came together. It shows how important good data quality is for building trust across the business.
Sarah
I agree completely. Let's meet again once the dashboard is live. Then we can put together a short summary for the whole data team.
Thomas
Sounds good. I'll send you an early draft of the summary later today so you can have a look.
Sarah
Perfect. I look forward to reading it. Well done, Thomas.
Thomas
Thank you, Sarah. Speak to you soon.
Advanced version
Thomas
Good morning, Sarah. Have you had a chance to review the audit report from last Friday?
Sarah
Yes, I have. I went through it carefully over the weekend. The team pinpointed the issue to the data conversion into CSV format.
Thomas
That aligns with my suspicions. They discovered that the numbers were being rounded excessively. The script was utilizing an incorrect number type, which resulted in inaccurate totals.
Sarah
Exactly. Furthermore, the 'contract_value_eur' column failed to retain four decimal places, which is a stipulation in our reporting standards, prompting the audit team to flag it.
Thomas
I’ve addressed that. The data is now saved in Parquet format rather than CSV, as Parquet preserves data types and structural integrity far more effectively.
Sarah
A prudent move. The audit team also insists on a secondary validation of the figures post-load. They want the system to reconcile the final figures against the originals.
Thomas
I’m currently implementing that. I’m leveraging a tool called Pandera, which scrutinizes every column and raises an alert if any anomalies are detected. Should the notional value deviate significantly, it will trigger an error.
Sarah
That’s excellent. We need to integrate that validation into our quality assurance workflow prior to dispatching the final report. Did you also revise the methodology for assigning risk levels?
Thomas
Yes. The updated script now pulls risk metrics directly from our risk tool, categorizing each product into one of three tiers: Low, Medium, or High. Legacy products will default to 'Unknown' unless we manually input the risk data.
Sarah
That should streamline the reporting process. The audit team also raised concerns regarding inconsistent timestamps, with some entries reflecting local time and others UTC.
Thomas
I noticed that as well. I’ve standardized all timestamps to UTC+0 in accordance with ISO 8601 standards. Each entry is now also assigned a unique identifier to facilitate tracking.
Sarah
That will certainly streamline future audits. They also emphasized that our report must explicitly document all data modifications and the rationale behind them.
Thomas
I’ve restructured the report format accordingly. There is now a dedicated section that details every significant alteration-such as table joins or data enrichment-complete with attribution and timestamps.
Sarah
Outstanding. One more point: the compliance team has requested a Power BI dashboard. It needs to display daily data updates, along with additional metrics like warning counts and confidence levels.
Thomas
I’ve already initiated its development. I’m using Azure Synapse to integrate the data streams. The dashboard will visualize daily changes and flag any outliers. I anticipate having it ready by the end of the week.
Sarah
That will provide much-needed visibility across the board. Have you had a chance to consult with the Data Governance team?
Thomas
Yes. I’ve shared a flowchart outlining each stage of our data pipeline, along with detailed descriptions of the processes and data inputs involved.
Sarah
Very thorough. Once we’ve completed these updates, we should conduct a stress test. We can introduce erroneous data to observe how the system responds.
Thomas
A sound strategy. I’ve set up a test environment in Snowflake that mirrors the production system, allowing us to run simulations safely.
Sarah
Excellent. Finally, we need to formalize everything. The audit team appreciated your glossary. Can we expand it to include error-handling procedures and contingency protocols?
Thomas
Absolutely. I’m collaborating with the documentation specialist to compile a comprehensive guide with step-by-step instructions and notes. We’ll also version-control everything using Git.
Sarah
Once that’s finalized, we should brief the front office. They need to understand how these changes impact their operations.
Thomas
I’ve already drafted a concise slide deck. It illustrates how our data cleansing directly influences their trading decisions, particularly regarding risk and capital allocation.
Sarah
That’s precisely the kind of cross-functional collaboration we need. Thank you, Thomas. We started with a minor discrepancy and have managed to drive a significant improvement.
Thomas
It’s been quite a learning curve, but I’m gratified by the collective effort. It underscores how robust data integrity fosters trust across the business.
Sarah
I concur. Let’s reconvene once the dashboard is live. Then we can draft a brief summary for the entire data team.
Thomas
Sounds like a plan. I’ll send you a preliminary draft of the summary later today for your review.
Sarah
Perfect. I look forward to it. Well done, Thomas.
Thomas
Thank you, Sarah. I’ll speak with you soon.
Check your understanding
1. How did Thomas resolve the data format issue to preserve data types and structural integrity?
Show answer
Thomas changed the data format from CSV to Parquet, which preserves data types and structural integrity more effectively.
2. What tool is Thomas using to implement secondary validation of figures post-load, and what does it do?
Show answer
Thomas is using a tool called Pandera, which scrutinizes every column and raises an alert if any anomalies are detected, such as if the notional value deviates significantly.
3. How has the methodology for assigning risk levels been revised in the updated script?
Show answer
The updated script now pulls risk metrics directly from the risk tool, categorizing products into Low, Medium, or High tiers, while legacy products default to 'Unknown' unless manually input.
4. What standard did Thomas apply to standardize timestamps, and what additional identifier was added?
Show answer
Thomas standardized all timestamps to UTC+0 in accordance with ISO 8601 standards, and each entry is now assigned a unique identifier to facilitate tracking.
Grammar practice (mixed)
Tenses
Sarah ___ the audit document over the weekend.
Show answer & why
reviewed · 💡 The specific past time marker 'over the weekend' requires the simple past tense.
Prepositionsself-check
The rounding issue occurred ___ the CSV intermediary export stage.
Show answer & why
during · 💡 The preposition 'during' is used to indicate that something happened within the time period of a specific process or event.
Idiomsself-check
The rounding error seemed minor at first, but it ___ in the final aggregate.
Show answer & why
piled up · 💡 The idiom 'pile up' means to accumulate or increase in amount, which fits the context of errors accumulating in a final calculation.
Conditionalsself-check
If the field had used float64, the system ___ the rounding errors.
Show answer & why
would have avoided · 💡 This is a third conditional sentence describing a hypothetical past situation; the structure requires 'would have' + past participle in the main clause.
Linking words/phrases
The output is ready for review, ___ the documentation still requires Sarah’s input.
Show answer & why
although · 💡 The conjunction 'although' introduces a contrast between the readiness of the output and the pending requirement for documentation.
Discussion (practise speaking)
How did the team address the issue of excessive rounding in the numbers during the audit process?
🤔 Think about a time you had to fix a technical error in your work.
Show sample answer
- The script was updated to use a correct number type.
- The data format was changed to Parquet to preserve data types.
- A secondary validation tool was implemented to check for anomalies.
Ask Phil: Practise describing a technical problem and its solution with the Pickle AI tutor.
What steps were taken to ensure consistency in timestamps across the report?
🤔 Consider how you handle time-related data in your own projects.
Show sample answer
- All timestamps were standardized to UTC+0.
- ISO 8601 standards were applied.
- Each entry was assigned a unique identifier.
Ask Phil: Practise explaining the importance of standardizing data formats with the Pickle AI tutor.
Why is it important to document all data modifications and their rationale in the report?
🤔 Reflect on how documentation helps in your current role.
Show sample answer
- It helps with future audits.
- It provides a clear trail of changes.
- It ensures transparency and accountability.
Ask Phil: Practise discussing the benefits of thorough documentation with the Pickle AI tutor.
How does the new Power BI dashboard improve visibility for the team?
🤔 Think about how dashboards can improve decision-making in your workplace.
Show sample answer
- It displays daily data updates.
- It shows warning counts and confidence levels.
- It visualizes daily changes and flags outliers.
Ask Phil: Practise describing the benefits of a dashboard in your role with the Pickle AI tutor.
Vocabulary
- data conversion
- ↗
reveal definition
The process of changing data from one format to another. “The team pinpointed the issue to the data conversion into CSV format.” - reporting standards
- ↗
reveal definition
Established rules or guidelines for creating reports. “The 'contract_value_eur' column failed to retain four decimal places, which is a stipulation in our reporting standards, prompting the audit team to flag it.” - data types
- ↗
reveal definition
Categories of data that determine the kind of values it can hold. “I’ve addressed that. The data is now saved in Parquet format rather than CSV, as Parquet preserves data types and structural integrity far more effectively.” - quality assurance workflow
- ↗
reveal definition
A systematic process to ensure products or services meet quality requirements. “We need to integrate that validation into our quality assurance workflow prior to dispatching the final report.” - risk levels
- ↗
reveal definition
The degree of potential loss or danger associated with an investment or project. “Did you also revise the methodology for assigning risk levels?” - data modifications
- ↗
reveal definition
Changes made to existing data. “They also emphasized that our report must explicitly document all data modifications and the rationale behind them.” - data governance
- ↗
reveal definition
The overall management of the availability, usability, integrity, and security of data used in an organization. “Have you had a chance to consult with the Data Governance team?” - data cleansing
- ↗
reveal definition
The process of detecting and correcting (or removing) corrupt or inaccurate records from a dataset. “It illustrates how our data cleansing directly influences their trading decisions, particularly regarding risk and capital allocation.” - data integrity
- ↗
reveal definition
The accuracy, consistency, and reliability of data during its lifecycle. “It underscores how robust data integrity fosters trust across the business.” - cross-functional collaboration
- ↗
reveal definition
Cooperation between people from different departments or areas of expertise. “That’s precisely the kind of cross-functional collaboration we need.”
Key phrases (useful expressions from the dialogue)
- pinpointed the issue Identified the exact problem or cause.
- stipulation in our reporting standards A specific requirement that must be followed in reports.
- structural integrity The quality of being complete and not easily broken or changed.
- raises an alert Triggers a warning or notification.
- default to 'Unknown' Automatically assigned a status of 'Unknown' if no other information is available.
- facilitate tracking Make it easier to monitor or follow something.
More Meetings & Discussions lessons
Addressing Board Concerns on Coastal Installation
A client follows up with a supplier after presenting initial product details to their board, which raises spec…
Addressing Competitive Pricing and Fulfillment Challenges
Clara and Sophie discuss competitive pricing pressures and inventory fulfillment issues to optimize Q4 sales p…
Addressing Late Reports and Budget Discrepancies in Programme Coordination
A founder discusses programme reporting delays, budget discrepancies, and workload management with a field coo…
Analyzing SEO and Social Media KPIs for Strategy Optimization
Alex and Jordan review website and social media performance metrics after a successful week of SEO and SMO cam…
Annual Performance Review and Budget Planning
An employee meets with their manager to review the first year's performance, discuss financial compensation, a…
Conducting a Systems Overhaul Needs Analysis
An IT consultant meets with an insurance CEO to identify pain points in legacy systems and plan a modernizatio…
Keep going
Practise this scenario live with Phil, and get a personalised lesson built from your own mistakes. Interactive in the browser, with a booklet to keep.
Book a lesson with Phil