What is Heuristics? Definition, Working, and Examples

Heuristics is an approach that steers an algorithm toward finding workable solutions for complex problems.

December 1, 2022

Heuristics is defined as a problem-solving or decision-making technique that uses minimum relevant information, past results, and experiences to produce a workable and practical solution for a problem in a reasonable period. This article explains the core principles underlying heuristics, its working, and some key examples in today’s computing world.

What Is Heuristics?

Heuristics is a problem-solving or decision-making technique that uses minimum relevant information, past results, and experiences to produce a workable and practical solution for a problem in a reasonable time. These strategies focus on providing quick results with an acceptable accuracy range rather than offering near-perfect solutions.

Heuristics comprises vital ingredients of the machine learning (ML) and artificial intelligence (AI) disciplines. It is a go-to approach when it is highly impractical to derive a solution for a problem by following a step-by-step algorithm. Moreover, as heuristic strategies look to provide speedy solutions rather than accurate ones, they are generally blended with optimization algorithms to improve the results.

Technically, all iterations are interdependent, implying that each level of a deep neural network is crucial in deciding which solution path to choose and which to discard based on their closeness to the desired result. Thus, the term ‘heuristics’ is synonymous with ‘short-cut’, since it does not employ resources to explore solution paths that do not yield acceptable results.

Heuristic methods in AI are based on cognitive science principles that revolve around ‘how humans think’. Moreover, heuristic algorithms in AI enable systems to produce approximate solutions rather than exact ones. Heuristics do not necessarily provide a cheaper solution. Instead, the ones that do not overestimate the cost of achieving the result are termed ‘admissible heuristics’. This is a crucial characteristic of heuristics that ensures the solution’s optimality. At the fundamental level, an admissible heuristic simplifies the original problem by reducing its constraints.

Although heuristic processes tend to find solutions or results that often work or are correct, they may only sometimes be right, provable, optimal, or accurate. However, decisions based on heuristics are usually good enough to solve small-scale problems and provide solutions in situations of uncertainty where complete information is unavailable. 

Heuristics rely on shortcuts to provide immediate, efficient, and short-term solutions that facilitate timely decisions for businesses. Analysts across industries use specific thumb rules that allow companies to address a problem and make decisions and judgments rapidly and efficiently. These include the process of trial and error, elimination, intelligent guesswork, past results or formulas, and even the analysis of historical data. However, in the computing world, a heuristic model acts as a rule of thumb to speed up and simplify decision-making processes in situations where there’s not enough time for careful consideration of all the aspects of the problem.

Advantages of heuristics

Heuristics facilitate the real-time monitoring of events while using fewer resources and minimizing the system’s load. It allows systems to handle big data and ensure a faster turnaround time for decisions on complex problems. Heuristic rules are vital for computing, cybersecurity, and risk prevention strategies.

Moreover, the approach plays a vital role in detecting newer variations of past problems and issues, combining larger datasets to identify connections between data points eventually. It allows the system to reach a definitive conclusion based on the configuration and helps choose a safe course of action that is void of risks.

Heuristics involve trade-offs when compared to traditional algorithms and decision-making methods. It prioritizes speed over precision, accuracy, or completeness of a solution. Moreover, it involves intelligent guesswork and even cuts corners to return solutions with more errors. Heuristic models rely on minimal calculations that may produce results prone to biases. However, the speed of the outcome overshadows the underlying shortcomings. For example, a heuristic-based system can immediately block financial transactions (online) based on blacklisted data points such as customer ID, contact no., email, browser hash, and so on.

Although heuristics do not offer an ideal mechanism to design solutions, one can consider the general drawbacks of the approach when configuring rules and setting up processes so that it enables you to choose scenarios where heuristics can be applied not only to speed up a task but also free up resources.

See More: What Is HCI (Human-Computer Interaction)? Meaning, Importance, Examples, and Goals

How Does Heuristics Work?

Heuristics generally refers to educated guesses that seem to deliver faster decisions than traditional approaches when dealing with problem-solving factors in the computing industry. Heuristic models typically perform the following tasks that enable faster decision-making:

  1. Analyze historical data
  2. Monitor real-time data frequently (i.e., 24×7)
  3. Compare data patterns in new and old data
  4. Make appropriate assumptions that fill unknown gaps in the data
  5. Trigger an action upon reaching a pre-set threshold for further processing

Heuristics are suitable for machine learning (i.e., white-box and black-box models), machine reasoning, and other related models that deal with diverse, big, and incomplete data. Heuristic-based techniques are popularly employed in trading & finance, cybersecurity, and fraud detection & prevention sectors. Moreover, they are being increasingly adopted by enterprises to advance their tech to enhance business productivity and efficiency.

Let’s understand the working of heuristics via a simple fraud detection example.

In fraud detection, heuristic models tend to terminate or block transactions by considering flagged data such as customer ID, cookies, contact details, or even specific action sequences in some cases. Let’s break down a simple use case.

  1. Assume a scammer or fraudster registers himself for an online gambling game app hoping to manipulate and abuse the ‘bonus packages’ offered by the gaming system.
  2. The fraudster has already tried this trick, though with a different device and contact details such as contact no. or user ID.
  3. In the second fraudulent attempt, although a few user credentials are different, the fraudster is still on the same IP address since he provided the same email and home address. Also, the individual is following the same steps on the gambling platform he used in his first attempt.
  4. Here, heuristics come to the forefront. The system uses a heuristic model to evaluate the newly entered data points picked up from the second attempt and match them to the shared data points associated with the first attempt. The model fills in the gaps and connects the new and old data points to arrive at a conclusion.
  5. The risk tolerance threshold is reached as the estimates tend to reveal the similarity between the first and second fraudulent attempts. As a result, the system blocks the fraudster from accessing or using it.

In this example, the IP address used in the second fraud attempt matched that of the earlier fraud event. Also, the fraudster used similar steps to exploit the system. Based on these parameters, the system assumes that the same person is trying to attempt another fraud.

Here, there’s a possibility that the result of risk analysis may just be a false positive. However, risk analysts have already considered several constraints and variables, such as ‘risk vs. reward’, to decide that it is better to proceed with false positives rather than face the consequences of false negatives. This implies that the analysts are okay with blocking legitimate users than missing the opportunity of blocking real fraudsters. Accordingly, the risk analysis team adjusts the risk tolerance threshold for the particular company or scenario.

In some cases, fraudsters can also put in user credentials, such as a house address that matches the address of a legitimate user. Some scammers tend to operate through a shared internet access spot that legitimate users use. These are the techniques used by fraudsters to play safely. However, with machine and browser fingerprinting, these data points are identified. From here on, the heuristic model starts finding connections and making assumptions essential for deriving conclusions.

See More: What Is Quantum Computing? Working, Importance, and Uses

Examples Of Heuristics

Heuristics is an inevitable and inseparable part of artificial intelligence. Simply put, it is a computer simulation of the human thinking process, used in situations no known algorithm can reach. Hence, heuristics are generally used in conjunction with optimization algorithms to enhance the overall efficiency of the desired results.

Here are some of the key examples where heuristics are routinely used:

1. Traveling salesman problem (TSP)

The traveling salesman problem refers to an optimization problem where a list of cities and the distance between each pair of cities is given. The task is to determine the shortest path to visit each city once and eventually return to the origin city. As multiple cities are traversed, the solution must also check cost (i.e., distance) and time complexity.

The TSP problem is generally considered an NP-hard problem (non-deterministic polynomial-time hardness) as producing an optimal solution even for a small- or moderate-sized dataset is a challenging task. As an alternative, a greedy algorithm can be used in this case to yield an approximate solution in a reasonably shorter period. This implies that the result approximates the optimal answer, which is a good-enough solution for the problem. The algorithm is a type of heuristics in one sense, indicating that the solution is close enough to the desired result. Although one can achieve solutions theoretically, an approximation is the best bet considering time constraints.

TSP is a combinatorial optimization problem having multiple applications in our daily lives. This includes vehicle routing, logistics (planning and scheduling), goods delivery, maritime industry, airport networks, public transportation networks in top-tier cities, and so on. TSP is a good example where heuristics play an important role.

2. Search optimization problems

Heuristics is known to make algorithms faster when handling specific ‘search optimization problems’. In step 1, heuristic rules tend to try out every possibility at each stage, which means executing a full-space search algorithm. However, the system can abort this search at any point if it recognizes that the current solution is worse than the best solution that has already been determined. Thus, heuristics helps optimize such search problems by initially trying out good choices or solutions while eliminating the wrong paths early.

Specific best-first search algorithms such as ‘A* search’ use heuristics to boost the algorithm’s convergence while keeping track of the correctness of the solution as long as the heuristic is admissible; for example, search engine optimization. Search engines help individuals find relevant information from millions of data sources. However, with such vast volumes of information on the internet, finding helpful content can be difficult. To make the process as swift as possible, search engines use heuristics to expedite the search process and ensure that individuals find relevant information in minimal time.

3. Heuristic search hypothesis

Allen Newell and Herbert A. Simon proposed the heuristic search hypothesis. In this hypothesis, solutions to complex problems are revealed as symbol structures. The symbol system then employs intelligence to solve the problem through a search. The process repeatedly generates, modifies, and restructures symbols until the created structure matches the solution structure.

Thus, each step invariably depends on the previous step. As a result, the heuristic model learns which paths to pursue and which ones to eliminate by verifying the closeness of the current step to the desired solution. Consequently, this process saves time and resources as some possible solutions may not be generated based on its measured unlikeliness to complete the solution.

In this context, a heuristic model fulfills its task by exploiting search trees. Rather than generating all possible solution branches in the initial stages, the model selects branches that have a higher probability of producing outcomes than other branches that are less likely to do so. At each decision point, the heuristic picks up the branches that produce acceptable solutions at each decision point.

4. Antivirus software

Antivirus software relies on heuristic rules to identify, detect, and isolate different forms of malware. It scans the software under consideration and looks for the code patterns or even behavioral patterns of the viruses with rules in place for different virus types. During the process, it is identified if the file or executing strategy reveals a particular code pattern or pattern of activities. As a result, it is inferred that the file is infected.

Moreover, with behavior-based heuristic scanning approaches, even self-modifying polymorphic viruses are traceable, unlike other simple scanning methods. Heuristic-based scanning can detect mutating viruses without needing to be detected previously. It can see suspicious new file codes or behavior patterns in real-time without needing to be noticed, analyzed, and labeled as an ‘xyz’ virus. As such, future viruses can be tackled by following the heuristics approach.

5. Knapsack problem (KP)

A knapsack problem consists of a group of items, each having a weight and a value. The task is to determine the total number of things to include in a combination so that the total weight of the item is less than or equal to a specific weight and the entire item value is as high as possible.

A heuristic model in the form of a greedy algorithm can be employed to solve the problem. The algorithm arranges items in descending order of value per weight and then inserts them in the sack. The technique allows the most valuable and heavy things to get into the pack first.

Such KP problems where heuristics play a vital role find applications across various fields such as machine scheduling, space allocation, asset optimization, home energy management, software resource management, optimizing power allocation to electronic equipment, network selection for mobile nodes, and so on.

See More: What Is Super Artificial Intelligence (AI)? Definition, Threats, and Trends

Takeaway

Heuristics in computer science refers to the ‘rules of thumb’ that algorithms use to determine approximate solutions to complex problems. As there’s too much information for systems to scan through before coming to a conclusion in a limited period, heuristic models prioritize speed over the correctness of the solution. However, it is crucial to consider that heuristic rules are specific to the problems you intend to solve, and their specifics may vary for every situation.

For example, let’s say you intend to apply heuristics to your algorithm designed to determine the number of moves a bishop can make on an 8×8 chessboard while traversing every square on the board. In this case, you can create heuristics that enable the bishop to choose a path with the most available diagonal moves. As you make a specific path, generating heuristic rules that allow the bishop to choose a path with the minimum available diagonal moves is better. As the available decision-making space is limited, solutions are also narrow, and as a result, they are found quickly.

Thus, with each definitive problem, you can design your own heuristic rules to finish a task in less time. This is a handy approach as some computationally complex issues may require years of computation to find the exact answer; however, you can produce an approximate result almost instantly with heuristics.

Did this article help you understand the fundamentals of heuristics and its role in computer science? Comment below or let us know on FacebookOpens a new window , TwitterOpens a new window , or LinkedInOpens a new window . We’d love to hear from you!

MORE ON ARTIFICIAL INTELLIGENCE

Vijay Kanade
Vijay A. Kanade is a computer science graduate with 7+ years of corporate experience in Intellectual Property Research. He is an academician with research interest in multiple research domains. His research work spans from Computer Science, AI, Bio-inspired Algorithms to Neuroscience, Biophysics, Biology, Biochemistry, Theoretical Physics, Electronics, Telecommunication, Bioacoustics, Wireless Technology, Biomedicine, etc. He has published about 30+ research papers in Springer, ACM, IEEE & many other Scopus indexed International Journals & Conferences. Through his research work, he has represented India at top Universities like Massachusetts Institute of Technology (Cambridge, USA), University of California (Santa Barbara, California), National University of Singapore (Singapore), Cambridge University (Cambridge, UK). In addition to this, he is currently serving as an 'IEEE Reviewer' for the IEEE Internet of Things (IoT) Journal.
Take me to Community
Do you still have questions? Head over to the Spiceworks Community to find answers.