Extreme programming practices that mitigate Technical Debt

Technical debt, first coined by Ward Cunningham(TD) [1, 2], can be described as the dichotomy between the best design that takes longer to implement and a quick design and implementation for short-term gain. The technical compromise between these two designs and extra development effort needed to make changes to the latter software system represents debt.

1.   What is Technical Debt?

Technical debt, first coined by Ward Cunningham(TD) [1, 2], can be described as the dichotomy between the best design that takes longer to implement and a quick design and implementation for short-term gain. The technical compromise between these two designs and extra development effort needed to make changes to the latter software system represents debt.

A popular metaphor is that of monetary debt. If TD is not repaid as soon as possible, “interest” is incurred — this relates to the effort needed to implement changes in the future. When TD is not addressed and systems are patched with functional and bug fixes, it leads to an increase of software entropy [3, 4], a condition that deteriorates over time. In some situations, when creating a proof-of-concept (POC), it is acceptable to incur some degree of TD. This, however, is known at design time and should be addressed when the POC evolves into an official project.

Another symptom of TD can be compared to the law of unintended consequences. More often than not, poorly designed systems have longer release cycles and have higher resistance to change. This is due to the fact that when necessary code changes are made to one part of the system, unintended changes occur and have to be addressed. This increases the time of the delivery cycle and drives the cost of change higher, often leading to more pressure, which in turn creates more TD.

Some causes of TD:

  1. Last minute emergency additions or omissions of functionality;
  2. Incompetent technical leadership and bad systems architecture;
  3. Efforts to minimize startup capital;
  4. Outsourced software engineering;
  5. Ignoring industry standards;
  6. Building tightly coupled code with modularity Loosely coupled code is easy to modify or change; and
  7. Non-existent test suite, continuous integration and deployment

 

TD manifests itself in the following examples:

  1. Countless branches of same code base, often for single customer releases;
  2. Database SQL stored procedure spanning a 1000 lines of code (LOC);
  3. Continuous version releases to fix bugs;
  4. Frequent patching of a legacy system; and
  5. Very large methods within

Figure 1: The author’s depiction of technical debt

2.    What is extreme programming?

Kent Beck [5] is widely considered as the creator of the XP software development methodology. XP is a frontrunner to many of today’s more modern software development methodologies. It is a lightweight, practical methodology with a focus on software quality and responsiveness to stakeholder requirements and change.

Intensive requirement analysis and extensive documentation are omitted from the process. Teams are small and focused, with a heavy emphasis placed on simplicity and short development cycles. User stories are a core doctrine of XP. Projects are initiated by the final users creating user stories describing the behaviour and functionality of the software system. Before any coding begins, functional testing of the requirements is conducted, with automated testing throughout the lifecycle of the project. Code is constantly refactored to reduce complexity, drive efficiency and adhere to standards. The result is extensible and maintainable code.

Good relationships between software engineers and stakeholders are also considered a cornerstone of XP as defined in the XP value system: communication, simplicity, feedback, courage and respect.

A software engineering team does not necessarily follow every XP practice. Self-organizing teams use and follow the XP practices that suit them at a point in time. As the software engineering team grows in maturity, so more XP practices are incorporated or omitted.

Given that most software systems are in a state of flux, XP adapts with the software system without being a technical barrier. XP manages this by having both technical and management practices. There are several XP practices, but for the purposes of this blog, the primary practices of XP are broken down in Table 1.

Table 1: Primary extreme programming practices

2.1  Modularity violations as a cause of technical debt

When code follows a good design and strictly adheres to standards such as the SOLID principals [6], changes to one module do not affect another module. A module can be described as a “unit whose structural elements are powerfully connected among themselves and relatively weakly connected to elements in other units.” [7]. This inherently creates an orthogonal design.

“Orthogonal design is the union of two principles, cohesion and coupling [8]”. To be more specific, loosely coupled code creates high cohesion, which makes responding to changes very easy and predictable. Positive side-effects of orthogonal design lead to clear and logical structure of relationships and create significant reuse.

Modularity violations occur when changes to one module unexpectedly induces changes to another module. This is not orthogonal design and a key indicator of TD. Changes to the code are not easily made and time estimation becomes inaccurate.

These violations must be identified and rectified by relentless refactoring, a cornerstone practice of XP. Strict and comprehensive coding standards must underpin the refactoring efforts of code, another important XP practice. The two XP practices complement one another and when used together lead to a direct and significant decrease in TD.

In the research paper Comparing Four Approaches for Technical Debt Identification, substantive findings are made by Zazworka et al. A very high correlation exists between Modularity violations and change prone classes, so much as 85% as shown in Table 2 below.

Table 2: Correlation between Modularity Violations and change prone classes [9]

2.2  Technical debt estimation variance

An integral function of software engineering is estimate development time for maintenance and new components. TD can have a significant impact on such estimations. Variances in estimation are depicted by the Cone of Uncertainty. Instead of being certain of the estimations given, TD creates increased uncertainty at the later stages of the lifecycle, where estimates should be more accurate (shown in Figure 2). This negatively impacts cost, productivity and project schedules.

Figure 2: Adapted Cone of Uncertainty impacted by technical debt [10]

Deadlines frequently being missed and so-called unforeseen changes to the code during the development cycle are indicative of time estimation variance due to TD. Accurately predicting development duration is not an easy task, TD makes this exponentially more difficult. Most estimates are given at the start of the phase, when TD is encountered estimates become inaccurate due to the extra work effort involved.

2.3  Measurement of Technical Debt

In the research recorded in Measuring Architectural Technical Debt [11], Kuznetcov defines TD as follows:

[TD] Principal – cost of eliminating TD (Code Debt, Design Debt etc.).

[TD] Interest probability – the probability that the type of TD will have an impact on the system (likelihood of imminent refactoring).

[TD] Interest amount – the extra cost incurred for addressing the TD (cost of modifying a module earmarked for refactoring as opposed to the cost of modifying after refactoring).

TD : { TDprincipal, TDinterest }

           man hours           work incurred           loss of productivity

TD can be expressed as an array where TDprincipal is independent of TDinterest. In keeping with the debt metaphor, the principal debt can be higher that the interest incurred or vice versa.

Various approaches are available to calculate the TD principal. Curtis et al. describes the TD principal value as a function of must-fix problems, the time required to fix, and the cost of the fix. Source code analysis provides actual counts and values calculated against input variables and the code itself. In the code, structural problems, modularity violations, LOC and duplication (to name but a few) all provide input for this analysis. TD can then be expressed as a product of this source code analysis. The principal amount of TD can be calculated using the high-level formula:

TDprincipal = Nmust-fix issues x Ttime required x Ccost incurred

2.4  Technical debt mapped to XP practices

All software systems develop some pain points over time. These pain points are based on the ISO 9126 [12] standard. (This standard has been revised by the ISO/IEC 25010 [13] standard.) A generic indicator of software quality can be defined by six characteristics: functionality, reliability, usability, efficiency, maintainability, portability [12].

These points are dealt with as they arise, and they are also indicators of TD. In Figure 3,  the author adapted the original diagram by Zazworka et al. [14] to reflect a mapping with the relevant mitigating XP practices.

By analysing the characteristics of each XP practice and variants of TD, a mapping can be made of the XP practices that effectively reduce the relevant form of TD.

Figure 3: Adapted Technical Debt landscape [14]

2.4.1  Coding standards – Directly influence the actual structure, composition and interaction of a software system at its lowest level. This can include guidelines and rules that the software engineer has to follow when writing code. This important practice touches on every part of the design, writing and testing of code.

2.4.2  Continuous integration – Facilitates in the feedback of system health and interaction in a continuous and predictable manner. As code is written and checked into version control, an extensive process of compiling, testing and deploying is started. Feedback is provided from all these sub-processes. Processes fail fast and provide immediate information.

2.4.3  Incremental design – Allows for the evolution and advancement of the software system in small manageable and measurable increments. Small releases are easy to scrutinize for TD. Also, small frequent releases are highly productive and stakeholders perceive a growing healthy software system.

2.4.4  Refactoring – Allows for the evolution and advancement of the software system on a code level and keeps the system at an optimal level of engineering excellence. If the code is frequently refactored, it becomes familiar to the software engineering team and TD is dealt with proactively.

2.4.5  Test-first programming – Software defects are kept at a minimum. The addition of new functionality is made simpler and more visible due to observable and trackable test results. As all the tests pass, the software system’s integrity is maintained and confirmed. As the system grows, the test suite grows too.

3.      Findings

Research indicates that there are various approaches to measure and indicate the extent of TD. This applies to both TDprincipal and TDinterest as described in Section 2.3. Only once there is a manifestation of a standard approach, can tools like SonarQube [15] come to fruition and be more scientific [11]. The SQALE[16] method has been widely adopted as a standard way of measuring and expressing TD. The adoption allows for increased feedback and evolution of the method.

Apart from the common causes of TD discussed in this paper, disconnect between technical staff and management is a major cause of project overrun and subsequently TD. This is exacerbated by the lack of further learning by middle management. Middle management also influences technology decisions and lack of understanding technology leads to an increase in TD.

When XP is implemented effective mitigation of TD occurs naturally. XP has had a significant impact on software engineering in the last decade. CI, Refactoring and Test-first programming have been instrumental in this, especially in the management of TD [17].

Each practice addresses one or more forms of TD [18]. Coding standards combined with refactoring and static code analysis directly increases code quality. Refactoring also aids in the writing of unit tests. Further, if done correctly, code will always be in a better state than before refactoring took place. Every new feature is built off a better base than the previous feature — this is true continued improvement. CI reduces the cost of integration and provides continuous feedback, and mitigates operational debt. Incremental design allows for the focus to be on functionality needed now, small increments and modularity counter design debt. Effective use of Test-first programming dramatically reduces defects and increases code quality, this mitigates code and test debt [19, 20].

In previous research [21] conclusive empirical evidence was provided in the written responses of interviewed software engineers. Applying a points system to the ranked XP practices provides a clear hierarchy of effective- ness. The Coding Standards XP practice is ranked first by a substantial margin. Refactoring and incremental design follows in rank respectively. Lastly, test-first Programming and continuous integration occupies the last two positions respectively.

Table 3: XP practices ranking

4.      Conclusion

The five XP practices are intrinsically linked in their ability and effectiveness in the mitigating of TD. The effect of following all these practices drastically reduces levels of TD. The power of these XP [22] practices lie in fact that they complement and follow on from each other. Most modern agile methodologies heavily incorporate these five practices, a testament to their importance.

In its simplest form: “Technical debt is the difference between what was promised and what was actually delivered” [23]. This difference can not only be very difficult to measure, but also difficult to find the root cause for the difference. TD will always be incurred at varying degrees of cost. An argument must be made that a software system is born from the code itself, not the idea of a software system. The code captures not only the core business idea or solution to the problem, but the reasoning and cognitive ability of a group of people, an amalgamation of intellect. Code must be guarded, maintained, optimized and treated with a level of respect. The empirical evidence presented substantiates that the five XP practices aid in this.

With time the measurement of TD and tools reporting on TD will increase in sophistication. There is no single solution to measure, report and reduce TD but rather a combination of measures and practices. The astonishing amount of research papers and commercial literature on the subject is proof of this.

Adapting to change is crucial to any software system evolving over time. Empirical evidence shows that this evolution can have negative consequences, but the practices we use to measure and mitigate the impact are also evolving in their effectiveness.

REFERENCES

[1]  W.Cunningham. “The WyCash portfolio management system.” ACM SIGPLAN OOPS Messenger , vol. 4, pp. 29–30, 1993.

[2]  Debt Metaphor – Ward Cunningham. URL https://www.youtube.com/watch?v=pqeJFYwnkjE. Last accessed: 1 October 2016.

[3]  Software Entropy. URL https://en.wikipedia.org/wiki/Software_entropy. Last accessed: 1 October 2016.

[4]  N. Ford. Evolutionary architecture and emergent design: Investigating architecture and design. IBM developerWorks. URL http://www.ibm.com/developerworks/java/library/j-eaed1/index.html.  Last accessed: 1 October 2016.

[5]  K. Beck and C. Andres. Extreme Programming Explained: Embrace Change, 2nd Edition. Addison Wesley, 2005.

[6]  Solid principles. URL https://en.wikipedia.org/wiki/SOLID_(object-oriented_design). Last accessed: 1 October 2016.

[7]  C. Y. Baldwin and K. B. Clark. Design Rules: The power of modularity. The MIT Press, 1999.

[8]  J. Coffin. Cohesion and Coupling: Principles of Orthogonal, Object-Oriented Programming. URL http:// www.jasoncoffin.com/cohesion-and-coupling-principles-of-orthogonal-object-oriented-programming/. Last accessed: 1 October 2016.

[9]  N. Zazworka, A. Vetro, C. Izurieta, S. Wong, Y. Cai, C. Seaman, and F. Shull. “Comparing Four Approaches for Technical Debt Identification.” Tech. rep., 2016. URL https://www.cs.montana.edu/ courses/esof522/handouts_papers/TDLandscape.pdf.

[10]  B. W. Boehm. Software Engineering Economics. 1981.

[11]  M. Kuznetcov. Measuring Architectural Technical Debt. Master’s thesis. URL www.ru.nl/publish/pages/ 769526/z-mscis-s4340132-mkuznetcov-2014-08-28.pdf.

[12]  ISO 9126 standard. URL http://www.iso.org/iso/catalogue_detail.htm?csnumber=22749. Last accessed: 1 October 2016.

[13]  ISO/IEC 25010 standard.     URL http://www.iso.org/iso/home/store/catalogue_ics/catalogue_ detail_ics.htm?csnumber=35733. Last accessed: 1 October 2016.

[14]  N. Zazworka. Technical Debt. URL http://www.nicozazworka.com/research/technical-debt/. Last accessed: 1 October 2016.

[15]  SonarQube. URL http://www.sonarqube.org/. Last accessed: 1 October 2016.

[16]  SQALE. URL http://www.sqale.org. Last accessed: 1 October 2016.

[17]  N. Brown, Y. Cai, Y. Guo, R. Kazman, M. Kim, P. Kruchten, E. Lim, A. MacCormack, R. Nord, I. Ozkaya, R. Sangwan, C. Seaman, K. Sullivan, and N. Zazworka. “Managing Techni- cal Debt in Software-Reliant Systems.” Tech. rep. URL https://pdfs.semanticscholar.org/f754/ db80f0e465cfcad4077c5703ff1cdfd8e902.pdf.

[18]  J. Holvitie, V. L. nen, and S. Hyrynsalmi. “Technical Debt and the Effect of Agile Software Development Practices on It – An Industry Practitioner Survey.” Tech. rep. URL http://conferences.computer.org/ mtd/2014/papers/6791a035.pdf.

[19]  C. Sterling. Managing Software Debt – Building for Inevitable Change. Addison Wesley, 2010.

[20]  J. C. Sanchez, L. Williams, and E. M. Maximilien. On the Sustained Use of a Test- Driven Development Practice at IBM. URL https://pdfs.semanticscholar.org/a00c/ 61b77e2df21b43d5e500341d5efec286c195.pdf. Last accessed: 1 October 2016.

[21]  C. Fourie. “EXTREME PROGRAMMING PRACTICES THAT MITIGATE TECHNICAL DEBT.” Tech. rep., School of Electrical and Information Engineering, University of the Witwatersrand, 2016.

[22]  Extreme Programming . URL http://www.extremeprogramming.org. Last accessed: 1 October 2016.

[23]  Escaping the black hole of technical debt. URL https://www.atlassian.com/agile/technical-debt. Last accessed: 1 October 2016.

by Carel Fourie

Human-centered design

Gone are the days when design used to be about aesthetic execution, when the main focus was to get as much work out the door as possible so that there was more time for more work — when brands spoke down to their consumers instead of speaking to them.

Gone are the days when design used to be about aesthetic execution, when the main focus was to get as much work out the door as possible so that there was more time for more work — when brands spoke down to their consumers instead of speaking to them.

Now most brands are waking up to the fact that we are living in an ever-changing, ever-growing, fast-paced world where the consumer has access to all kinds of information literally at their fingertips. And design plays a crucial part in the world we live in today. From the food we eat to the information we choose to consume online, design is everywhere.

Brands are cottoning on to the fact that their customers are more informed than ever before, and so big brands like Apple, Google, Uber, Airbnb, Facebook etc. are placing the consumer at the core of everything they do. This means that brands are now allowing their customers to decide on the type of content they want to consume and then designing for that. In the product design space this is so important — keeping the consumer at the center of everything that you do for a better experience.

Human-centered design is all about developing good relationships with your customer by delivering a high quality product that through prototyping and testing results in an emotional connection between the customer and the product.

                              Fig. 1 The human centered design pyramid (source: Giacomin, 2014)

Good design needs to be able to answer a set of key questions to facilitate this connection.

  • Who is the consumer?  Does the design reflect the user characteristics?
  • What are the consumers’ goals when using the product?
  • What is their experience when using the product?
  • What are the goals of using this specific product or service?
  • When and how does the consumer interact with the product design?
  • What do consumers think about the product or the design?
  • Why does the consumer want to use this product or design?
  • http://www.designorate.com/characteristics-of-human-centered-design/

 

Consumer feedback is key in ensuring that the design continually improves. And so, unlike before, the design process is never complete. Especially in the product design space, it is very important to keep iterating and making your product better with each iteration. Part of achieving that emotional connection with the product is about designing experiences as opposed to designing products.

The commonly used tools in building a human-centered approach are:

  • Personas;
  • Scenarios; and
  • Use cases.

 

Persona: This refers to creating fictional character that could potentially interact with your product. This usually includes their age, race, gender, location etc. Basically, it’s the target audience.

Scenarios: This would be the possible scenario of the persona using your product.

Use cases: This refers to the feedback gathered from the Persona through the Scenarios

It’s time that brands start immersing themselves in the worlds of their consumers if they want to remain relevant.

by James Mokhasi

Design Indaba made me do it –

This was the mantra for the 22nd annual Design Indaba conference, hosted by the beautiful city of Cape Town at the Artscape theater.

This was the mantra for the 22nd annual Design Indaba conference, hosted by the beautiful city of Cape Town at the Artscape theater.

The Design Indaba Conference has grown to become one of the world’s leading design events and hosts more than 40 speakers and 2 500 delegates. It draws creatives from all spheres and industries to come together under one roof to share knowledge, inspire and to collaborate with one another.

We talked, mingled and networked; filing our inspiration tanks. There were graffiti artists, dj’s, musicians, sculptors and various sponsor pop-ups and activation units, inviting us into this world of endless possibility and creativity.

Contrary to current perception, Design Indaba is not a conference ONLY for creatives – it is for everyone, from any field of expertise that would like to ignite their senses and intrigue their minds. It’s a jam packed 3 days and I believe that there is something that will speak to anyone’s core. This year was my first Design Indaba and it was a truly immersive experience, exceeding all my expectations.

The main highlight for me, wasn’t the skill or talent of all these amazing people (even though that was incredible) – but rather their thinking, this really stood out to me; they took us on a journey through the lens and into their magical minds!

Ultimately, Design Indaba wants to change the thinking of the world, one conference at a time, one creative at a time, and one business at a time.

It will take a generation of creative thinkers and implementers to see a turnaround. Design Indaba’s primary aim therefore is “to advance the cause of design as a communication fundamental, a business imperative and a powerful tool in industry and commerce, awakening and driving a demand for investment in intellectual capital”.

Investing nearly two decades in this vision, Design Indaba has championed the creative revolution. Here are some of my highlights from the 3-day event (content supplied from the Design Indaba weekly mailer):

The enchanted forest – Can beauty redeem us?

We were welcomed into the Design Indaba Festival 2017 through an enchanted forest of massive tree sculptors that were beautiful and surreal.

These tree sculptures were on exhibition the entire conference and created a magical ambience to the atmosphere in the festival court yard. I felt like I was walking around in a world that was a mash-up of the movies, Labyrinth and Alice in Wonderland (Tim Burton version).

Read more >

Capturing Cape Town’s scent with Kaja Solgaard Dahl

The thank-you gift for the festival this year was created by this designer, Kaja Dahl, she is fascinated with creativity that uplifts our experience and affect the senses directly.

Her process and the end-product is captivating and just incredible. She truly did capture the scent of Cape Town –whimsical, fresh, enlighten, yet eccentric.

Read more >

Masters in the art of freestyling it

One of my main highlights of the festival was the amazing group called Freestyle Love Supreme. They would wrap up each day with freestyle rap and beat boxing. They were so entertaining and funny, I laughed so hard that may face hurt.

The Design Indaba team chatted to Freestyle Love Supreme ahead of their Design Indaba daily wrap ups and once-off performance on the Thursday at Nightscape.

Read more >

 

 

Swahili launches on Duolingo

At Design Indaba 2017, Luis Von Ahn launches the first African language course on Duolingo. The audience went wild when he told us, he then went on to say that the second African language they will be launching will be Zulu. We can’t wait to see more African languages on this amazing app.

Read more >

Arch For Arch: A coda for Design Indaba Festival Day 3

The spectacular finale of the 2017 Conference and a tribute to Archbishop Desmond Tutu. It was a great honor and privilege for me to be a part of this amazing ceremony and to hear the incredible and humble, Archbishop Desmond Tutu talk. It was a great way to end the amazing festival, I left feeling inspired

Read more >

Thank you for the wonderful experience and we are looking forward to where they go from here.

So, if you think that design indaba isn’t for you – think again. Book your ticket for next year and immerse yourself.

by Mari-Liza Monteiro