How do we avoid technical debt when adopting AI? Insights from the Agile Tech Town Hall

Posted July 7, 2023 in Uncategorized by

A few weeks ago, we hosted a Tech Town Hall featuring Michael Wharton from KungFuAI and Sukant Ghosh from TestingAI, who shared their expertise on adopting artificial intelligence (AI). One of the key challenges discussed was the accumulation of technical debt when implementing effective AI solutions. Starting your AI journey on the right foot is crucial, and platforms like Hugging Face offer a solution. In this blog post, we will explore how leveraging Hugging Face can help businesses avoid technical debt and achieve success in their AI initiatives.

Understanding Hugging Face:

Hugging Face is a renowned open-source platform that provides businesses with access to a wide range of pre-trained models, APIs, and tools. By utilizing Hugging Face, businesses can kickstart their AI journey without accumulating technical debt. The platform offers a simplified implementation process, enabling rapid prototyping and reducing the need for extensive custom development. You can find more about them here: https://huggingface.co/.

Avoiding Technical Debt:

During the webinar, Michael Wharton, a Machine Learning Principal at KungFuAI, mentioned an article called “Hidden Technical Debt in Machine Learning Systems.” This article provides clear insights on how organizations can prepare for the challenges they may face when launching machine learning environments.

Here are the key insights from the article and practical steps to take when evaluating your next AI initiative:

  1. Technical Debt in Machine Learning (ML) Systems: Developing and deploying ML systems is relatively fast and cheap, but maintaining them over time can be difficult and expensive due to the technical debt incurred during the process. Technical debt refers to the additional rework caused by choosing an easy solution now instead of a better approach that may take longer.
  2. Erosion of Boundaries in ML Systems: ML systems tend to erode strict abstraction boundaries, significantly increasing technical debt. This erosion can occur due to factors such as entanglement (mixing of signals making isolation of improvements impossible), correction cascades (creating dependencies by learning a model that takes another model as input), and undeclared consumers (systems that silently use the output of a given model as an input to another system).
  3. Data Dependencies in ML Systems: Data dependencies in ML systems can contribute to technical debt and may be more difficult to detect than code dependencies. These can include unstable data dependencies (input signals that change behavior over time) and underutilized data dependencies (input signals that provide little incremental modeling benefit).

Based on these insights, here is a high-level guide for tech organizations:

  1. Awareness and Management of Technical Debt: Understand the concept of technical debt and its implications in ML systems. Make strategic decisions about when to incur technical debt and plan for its servicing. This could involve refactoring code, improving unit tests, reducing dependencies, and improving documentation.
  2. Maintain Abstraction Boundaries: Strive to maintain strict abstraction boundaries in your ML systems. This could involve isolating models and serving ensembles, focusing on detecting changes in prediction behavior and avoiding the creation of correction cascades. Also, ensure that all consumers of a model’s output are declared to avoid hidden dependencies and feedback loops.
  3. Manage Data Dependencies: Be aware of the potential for data dependencies to contribute to technical debt. This could involve creating versioned copies of unstable input signals and regularly evaluating your models to detect and remove underutilized data dependencies.

You can access the full article here.

Cost-Effective Solution:

Michael and Sukant both mentioned the cost-saving aspect of starting your AI journey with Hugging Face. They highlighted how solopreneurs and entrepreneurs can now implement AI themselves without significant upfront investments. Hugging Face’s open-source community support and the availability of free options allow businesses to experiment, iterate, and learn without breaking the bank.

Access to GPU Power and Distributed Computing:

Hugging Face provides businesses with the ability to access GPU power and distributed computing, which are crucial for running AI models effectively. By utilizing Hugging Face’s platform, businesses can rent GPUs on-demand, eliminating the need to invest in expensive infrastructure. This accessibility empowers businesses of all sizes to harness the computational power required for AI without incurring substantial costs.

Ensuring Success: Real-World Case Study

Michael Wharton shared insights from his experience, stating that his company regularly uses Hugging Face for the majority of their projects at KungFuAI. Hugging Face’s open-source community and its collection of tools and models have been instrumental in its success. The collaborative nature of the platform allows for continuous improvement, ensuring that the best models and resources are available to users.

We appreciate the insights from our guests and the knowledge they shared with us. We hope this information helps your team make the best decisions and avoid unnecessary technical debt.

Tags:
, , , ,
Print page