We’re committed to tackling the cost of living crisis in the UK

We’ve all seen it in the news… the cost of living is rising across the UK and the impact on young people is worrying. We’ve teamed up with our new charity partner MyBnk — who help young people to take control of their finances — to better understand what the financial world means for young adults right now.

Here’s what they’re thinking…

With fuel costs set to hit all-time highs, the coming months are almost certainly going to be a challenge for many people. And while all sectors of society are going to be affected by rising living costs, young people will be particularly hard hit.

Almost 50% of young people have what we call ‘low financial resilience’ which means they are unable to absorb the shock of a random bill or unexpected expense. In addition to this, 84% of young people have received no financial education at all, an increase of 17% since the pandemic. This means that a whole generation of 18-year olds are entering independent living without all the information and skills they need to successfully live alone. Combine all of this with an unprecedented global living costs crisis and we have the potential for a disaster amongst a large proportion of our society.

At MyBnk, we’re working hard to reverse this trend and build a society of well-informed, financially aware young people who are able to navigate all the systems they need to stay safe and well, no matter what their other challenges might be. We believe that a well-informed young person makes better financial choices, better understands their responsibilities and generally engages more with their money issues instead of burying their head in the sand.

We bring the world of money to life — through engaging materials, expert trainers and programmes of work tailored to all age ranges, we start to encourage people to talk about their money — breaking the taboo that you shouldn’t speak about money with strangers and therefore modelling that asking for help from creditors, or other advisory services, is normal and ok.

We make sure our work is relevant to the learners at the life stage they’re at, and that we’re clear and understood by young people when exploring complex topics such as taxation or the pros and cons of borrowing.

But we all have a collective responsibility to understand the challenges young people are facing as they begin to enter independence. Opening money conversations with young people means that they are more likely to ask for help when they need it — whether it’s because they don’t understand what they’re hearing in the news, or because they have made a money mistake and need some support.

Together with Compare the Market, we will start to break down barriers for young people, helping them to understand the current news and what a cost of living and energy crisis may mean for them.

The media is full of negative stories about how much today’s situation will hurt pockets up and down the country — and whilst this is true, we believe that early intervention, along with the right approach, can help our learners to be more confident about their situation and the challenges they face, preparing them for the turbulent road ahead as we face economic uncertainty.

So, what can you do?

– Book money management programmes for your young people. Our young adult money workshops The Money House and Money Works focus heavily on bills and living costs to help 16–25 year olds transition into independence. Available virtually and in-person. You can request sessions here.

– Signpost young people to our free resources: We have a dedicated website section aimed at 16–25 year olds. We’ve also just launched season two of our online webcast, focused on the energy crisis.

– Start the conversation! Talking money is essential to break the taboo around the topic and encourage young people to seek help for financial issues. Tackling the topic at home or with the young people you work with is a great way to build their confidence and encourage further conversations.

Compare the Market are partnering with MyBnk to make great financial decision making a breeze for the next generation. The partnership will reach 70,000 young people across the UK with expert-led financial education.

Take a look at this video to find out more about our work together.

Young people receiving financial education

Meerkat Your Skills

Large organisations’ reliance on data and analytics is becoming increasingly visible in the workspace and the skillsets required to analyse data through innovative methods are vitally important. Unfortunately, many charities and social programmes don’t have access to these types of skillsets.

comparethemarket.com (CtM) is acutely aware of this, and as such, many of our data analysts partake in Meerkat Your Skills — part of our CSR programme, aimed at using our skills to support good causes. The CtM Data team has many capabilities such as engineering, Business Intelligence, analytics, insights, optimisation, and data science. These teams work together to provide many data solutions to the business. CtM prides itself in using data to make better decisions. Modern technologies such as PowerBI and Databricks are entrenched in the business.

Our most recent event was held in December, where different teams from around the business including Data, Tech and Marketing supported a couple of charities and shared their expertise. CtM teams made the training and upskilling for each charity bespoke — to ensure it was relevant and actionable.

The Data team engaged with a charity called The Kite Trust, which supports the wellbeing of LGBTQ+ young people within Cambridgeshire and Peterborough. The ambassadors scoped out the requirements and completed project briefs. We learned that The Kite Trust needed help with understanding their geographic footprint and where there was opportunity for more group sessions.

The workday was successful and consisted of an initial meeting with the Data team volunteers and members of The Kite Trust to ensure the volunteers had enough context and understanding of the charity. After getting a good grasp on the data available and the project briefs, the volunteers set off working on their projects.

The team of analysts used mapping algorithms and spatial analytics to determine where access to community support groups was limited. These insights, using tools such as Databricks would have been difficult to gauge without a team of analysts. The Kite Trust will use these actionable recommendations to determine where support groups need to be enhanced or expanded to.

It’s not news that companies engage with community outreach programs. Holding events such as Meerkat Your Skills is very rewarding and leads to positive gains for both the charities and employees. This was a great opportunity to enhance the teams’ skills — working on a new dataset, completing different types of projects, and collaborating with members of various teams within Data. We enjoyed the experience and learned from the charity and are excited to do this again in the future.

Implementing a new data science and analytics platform Part 2

Recap.

We went through a non-exhaustive list of requirements for a good data platform, that we used to shortlist two solutions for a POC: Databricks and AWS Sagemaker.

In the part 1, I introduced our journey towards the implementation of a Data Science and Analytics platform. I explained that a data driven company needs to consider many aspects, from hiring good talent to investing in a new data platform.

Databricks is a software platform that helps its customers unify their analytics across the business, data science, and data engineering. It provides a Unified Analytics Platform for data science teams to collaborate with data engineering and lines of business to build data products. Find more details here.

Amazon SageMaker is a fully managed machine learning service. With SageMaker, data scientists and developers can quickly and easily build and train machine learning models, and then directly deploy them into a production-ready hosted environments. Find more details here. This was already available internally, as CTM uses AWS as cloud solution.

POC ran for a duration of a month, during which we assessed the functionalities of both solutions, and validated against each other as well as the current environment where relevant.

Methodology

The POC was divided into 2 main parts:

Architecture & devops assessment.

End to end testing.

Architecture & devops assessment

In this part, the focus was on the platform deployment and administration. We created an isolated AWS account, identical to the main account we use for our daily tasks. We then went ahead with the deployment of Databricks, which we found straightforward. The tests were evaluated against the following categories:

Deployment: How easy it is to deploy Databricks within AWS.

Administration: What features are available for the platform admin, how effective they are.

Tools & Features: Are the available tools capable of covering all our daily tasks.

Performance: Query performance, job performance.

Integration with external services.

End-to-end testing

This is where the solution has been tested in much finer detail, developing, and productionising a Machine Learning model with Databricks and Sagemaker.

Since Databricks casts a wider net than just Machine Learning applications, we have arranged for a 2-day Hackathon that involved various teams within the Data Function to go over a scripted task list, predefined by the representatives of each team (insights, Analytics, Data Science, etc).

This part was evaluated using a scorecard that rolled up into various categories such as:

Productivity & Workspace: ease of use, platform performance, stability of the environment.

Collaboration: Collaboration with other users, sharing results and dashboards.

Analytics: Data manipulation, visualisation, and data export.

Data Science: machine learning lifecycle management.

Note: List above is not an exhaustive one, just a high-level overview

Each team member was expected to score various tasks under each category. These were then discussed, to understand the reasons behind them and averaged where relevant to get an idea of which solution was preferred by the team. (Databricks, Sagemaker or current way).

Implementing a new data science and analytics platform

How do you choose a strong solution for your business? Which platforms are best? There’s lots to consider:

Be more data driven! This is a sentence we hear more and more. The boom of big data technologies have opened the doors to possibilities we could never have imagined before. The affordability of these solutions makes advanced analytics and data science available to all.

However, having a strong data science and analytics foundation requires a lot of aspects to be taken into consideration and investments to be made:

Hire new talent.

Review the internal technology stack and potentially invest in new technologies.

Put in place a proper governance around data related activities.

Compare the Market started this journey many years ago, by hiring data specialists (data engineers, data analysts, data scientists etc) and implementing new infrastructures (Hadoop initially).

This has led to the rapid growth of our data activities, with many positive results (massive processing of unstructured data, machine learning at scale).

However, a few years ago we decided to build on this by implementing a unified data science and analytics platform as this was easier to maintain, more cost effective and flexible for the work we were doing.

This was a long project, but after 10 months of work, it is now complete!

Over the course of a series of blog posts, I’ll share our learnings from the implementation.

I’ll cover:

Initial decisions.

Proof Of Concept.

Architecture design.

Implementation & Onboarding.

Note: What we are sharing in this blog is not THE way to implement a data science and analytics platform, but a solution that was fitting our context.

The problem

Although the business had invested in many data tools, there was no enterprise platform in place for delivering advanced analytics and data science. Given the growing size of the team, and the amount of incoming projects, we decided to look for a solution to enable collaboration between the team members, and allow to ingest new projects.

A few of our requirements were:

Scalability: Not just kit wise but also scalability of the team to take on more projects including upskilling, onboarding, collaborating etc.

End-to-end functionality: Ability to tackle various tasks on the platform, through standardised methods/kit, without depending on external resources.

Collaboration: Facilitate joint work on a project.

Skills gap: A platform providing skills stacks able to cover the major roles (from data analytics to data science to machine learning engineering).

PII / Sensitive Data controls: Meet the data governance and security requirements.

As a result, we have reviewed and engaged several vendors to explore market offerings and find a suitable partner to help us deliver the new platform.

We have done extensive research on the vendors listed in Gartner’s Latest Magic Quandrant for Data Science and ML Platforms and added some others we have interacted with during the last couple of years.

Some of the leaders in the magic quadrant were ruled out due to:

Op model: Proprietary software/licensing with high pricing, vendor lock-in and/or skew to on-premises infrastructure that would be inflexible to changes in our internal infrastructure.

Performance on key capabilities such as collaboration, advanced analytics & ML Ops and scalability.

Finally we landed on two options to run a POC:

AWS Sagemaker: Already available solution as CTM uses AWS as cloud solution.

Databricks: For the list of features it provides.

The POC ran for one month during which we have developed and tested functionalities available on the platforms. This is the topic of the next series, where we will see how we organised the POC to be sure to have an objective result and make an informed decision on the way forward.

Breaking a bias shouldn’t be an afterthought

At Compare the Market, we are proud of every one of our employees for the value they add to our business and working with bias in our teams is not something we will tolerate. With International Women’s Day in mind, we caught up with Dimple Dalby, one of our amazing Engineering Managers, about her experience of bias in her career and what advice she would give to the next female generation to keep stamping this out in the workplace.

What does International Women’s Day mean to you?

It’s a day to celebrate how far we have come, everything we have accomplished and to remind ourselves that the journey is not over! There is still more work to do so that the younger generation, and the ones after that, have a better and unbiased world to step into.

What does bias mean to you?

Bias to me is simply discrimination based on factors such as gender, race, age and ethnicity. It happens when people are not ready to accept anything — ideas, thoughts or actions — that is outside of the norm for what they are used to. And for us as women, we have centuries of biases to break and now is our chance to set up a new and equal norm for ourselves where we challenge the everyday inequalities.

Have you come across bias in your career?

Yes, of course… many times!

And while sometimes it has happened directly to me, I have also been a witness to it with people I have worked with. Either way, it’s not a nice place to be and can make you feel angry, inadequate and in the worst case, helpless.

The two areas where I have come across a strong bias in my career are:

Promotion to a leadership position.

Obtaining equal pay.

There were a few years in my career where I felt I had hit a brick wall as I would get promoted to a Senior Engineer but got constantly knocked back when applying for a lead role.

And this happened despite me leading many major projects — when it came to officially being given the role, I wasn’t even considered. I was never offered a solid reason why, just told that I was not ready and that it takes women many more years to gain the confidence to be able to lead compared to men. I wasn’t offered a path to train myself to be ready either.

It wasn’t until I joined Compare the Market that I got my first break into a lead role. Not only was I trusted to be the lead of a very important team, I was also given six months of leadership training to help me grow in my role and fill any gaps I may have had.

And in addition to this, I had many senior leads reach out to me offering support so while the journey was not always an easy one, I always felt that any help I needed was just an ask away.

How have you challenged that bias?

Challenging didn’t always come naturally to me and because some of these biases are so deeply ingrained in our society, challenging them can feel daunting.

But when I did finally have the courage to challenge, it wasn’t always handled productively or professionally to begin with.

Being overlooked in your career, based purely on your gender, can make you feel angry and anger can lead to a lack of clarity in your thoughts followed by unprofessional reactions that you may later regret.

When faced with bias at work, whether it was finding out that all my male colleagues were paid far more than me or being unfairly overlooked for a promotion, I have always tried to step outside the situation and analyse it.

Often my first reaction has been to feel angry which I think is ok and justified. But to channel it well, I wrote down my thoughts about the situation and how I would like to respond to avoid explosive or unprofessional reactions.

A few top tips that I have learnt along the way are:

Good and effective communication is key when tackling a sensitive issue.
Be direct and never be afraid to speak your truth and voice your concern.
Keep a record of the communications you have had.
Agree on an action plan with all parties involved.

In situations where it repeats itself and you feel it’s a much bigger situation than you can handle yourself, get the HR involved.

What advice would you give to the next female generation to help break the bias?

You are not alone in this even though it might often feel like that when you are a minority.

Don’t feel afraid to challenge the biases that you face as that is the first step towards change.

Only when enough of us challenge can it get highlighted and get the traction that it deserves. Being subjected to bias can make you feel under-valued, leaving you questioning your own confidence.

Don’t let this deter you from what we are all looking to achieve #BreakTheBias.

What advice would you give to employers?

Breaking biases should not be an afterthought.

Introducing unconscious bias training is only the beginning. This needs to be followed up with measuring the impact and outcomes that it has on employees to form the basis for longer-term planning and investment.

And as with everything, transparency is key. Always remain open and honest about criteria relating to areas such as internal promotions and hiring for new roles.

‘Tipping’ Open Source at Compare the Market

At Compare the Market we encourage and support self-organising communities. One such community which formed was the open-source group. This was established from a desire to give back to the wider open-source community from which we gain so much.

The open-source group in Compare the Market is passionate about ensuring that open-source projects receive reciprocal benefits when we use them. One initiative that they have recently introduced is the open source donation process. This process exists to allow teams to make financial donations, or “tips”, to the open-source projects on which their software relies. Through this process anyone in the company can request a donation to a project, which triggers a review and prompt payment when approved.

By giving something back, we help keep the open-source community healthy and the software we depend on maintained. It is also a great way to recognise the hard work that people have put into these projects.

With the successful set up of the tipping process under their belts, the open-source group is now looking at other areas where they can help. They are currently setting guidelines around contributing to open-source projects; making it as easy for our engineers to request changes to open-source projects as it is for them to inner-source work. They’re also looking to support and encourage our engineers to open source our in-house projects. In the course of our work, our engineers often create tools and applications which can be used more widely and we want to make it easy for our engineers to do more of this.

The open-source community in Compare the Market is very ambitious and determined and I’m sure we’ll be seeing lots more great ideas come out of that group.

How we reinvigorated our engineering career paths at Compare the Market

At Compare the Market we have been very lucky to have incredibly talented people. With my obvious bias, none more so than reflected in our engineering department where our engineers have grown from a single team 15 years ago, to more than 250 engineers today.

In that time, we have surmounted many challenges; pushing the boundaries of our technical estate in moving from a monolith to a decoupled microservice architecture, to a selectable-service based offering and now evolving once again to move to a truly customer outcome focused way of working.

Our engineering career path, however, had not kept pace with the mammoth changes taking place in our people or our systems. It was time for a refresh.

Why we needed a change

As our organisation developed, our estate expanded, and our people grew in number and in expertise. We added more roles, ensuring that our people had the space to grow.

These roles were left to be shaped largely by the people who moved into them and there were problems with our role structures, which were not necessarily reflective of how the whole organisation was structured. Unsurprisingly this led to people with the same titles doing vastly different jobs where responsibilities were not clear.

In Compare the Market we have done an amazing job at creating very high functioning, highly effective teams who work incredibly well. We promoted focus and independence within the teams but, as we become more collaborative across our engineering community, our engineers are able to move around more. Having consistent roles became ever more important and having clear guidance on how our engineers level-up is critical.

This organic evolution of our progression paths threw up some challenges. Within the individual team unit, the most senior role, that of Technical Lead, was also responsible for much of the day-to-day management of the team. Not all our engineers had the inclination to take on management responsibility. Similarly, the progression from Tech Lead was a move to Application Architect which was an individual contributor role and many Technical Leads really enjoyed the joint responsibility of technical and team management and didn’t necessarily want to move back into an individual contributor role to progress.

Finally, our architecture function was a separate entity to our engineering department which can sometimes create an artificial divide. The introduction of the Application Architect role a few years ago was the first step in bringing architecture closer to the engineers and we wanted to continue down the path of aligning these two functions.

What we wanted from our new career paths

Photo by Francesco Gallarotti on Unsplash

When we set out to develop a progression path that worked for us, there were a few things that we made sure to keep front and centre of our thinking.

The ability for engineers to choose whether they wanted to take an individual contributor or management track was paramount — but we needed to ensure that we didn’t lock them into any one path and that engineers could make the next step that best suited them and their aspirations. We wanted to allow people the flexibility to move along and between different paths in the ladder.

We needed to ensure that there were clear guidelines and an unbiased framework in place around each role. This was to aid with progression, performance reviews and to ensure consistency in that role across all our teams.

We also wanted to reframe our most senior technical roles and ensure that they were not seen as a separate ‘architecture’ department but were part of our engineering community. These highly aligned engineering multipliers now work in constant collaboration with our teams, guiding and leading them through technical challenges and step changes. We wanted our titles to reflect that relationship.

What we did

We created a dual track system for engineering — Individual Contributor (IC) and Engineering Management (EM).

The individual contributor track in Compare the Market is divided into two distinct parts. Firstly, we have the execution roles, that of Junior Engineer, Engineer and Senior Engineer. These roles are responsible for the development of software within the scope of a ticket, product, or team. Secondly, the Staff and Principal Engineers are our scale roles, who use their experience and skill to scale our community of engineers, typically within the scope of a team, a set of related teams or the whole engineering department.

The engineering management track perfectly combines people management with technical expertise. Our Engineering Managers are technical leaders who guide decisions on technical approach as well as mentor the professional development of their direct reports. As these leaders move up through the management track, they increase their scope and impact, making decisions that affect the whole engineering department and leading all of us from the front.

The move to engineering management may require the learning of a whole set of new skills and competencies: how to effectively coach, mentor and lead people is no small feat. Similarly, the move to Staff Engineer is a large step requiring increasingly complex competencies. There needs to be a deep understanding of architecture and design across the breath of the organisation, coupled with collaboration and cross-organisational expertise. We therefore allowed for the expansion of the responsibility of the Senior Engineer to ensure that their work is interesting and that they are continually challenged as we don’t expect every Senior Engineer to want to be a Staff Engineer or an Engineering Manager. However, it also allows our Senior Engineers to gain exposure to the skills and experience needed to take the next step should they wish to do so, whether on a management or IC track.

We also reframed our Application Architect and Solutions Architect roles to Staff and Principal Engineer respectively, bringing them into our engineering community. This reflected and formalised the close connection that had they already created with the engineering teams they work with.

We have many distinct types of engineers in Compare the Market; software engineers both front end and backend, data engineers, software engineers in test and cloud platform engineers who all work across disparate languages, platforms, and skillsets — but who all follow this similar progression path.

What next

We are now live with these new paths and if this sounds like the type of place you’d like to take the next steps to grow your career — take a look at our open jobs.

We are also working toward the creation of a capability framework which will provide detailed descriptions of the expectations of each role to provide an unbiased aid in progression planning, performance reviews, feedback and hiring.

We’re still working on embedding these new roles and pathways and I have no doubt that there will be lots to learn and adjust along the way but I’m confident that as a community, our engineers are, as they have always been, up for the challenge.