problem solving programming

How to Solve Coding Problems with a Simple Four Step Method

I had fifteen minutes left, and I knew I was going to fail.

I had spent two months studying for my first technical interview.

I thought I was prepared, but as the interview came to a close, it hit me: I had no idea how to solve coding problems.

Of all the tutorials I had taken when I was learning to code, not one of them had included an approach to solving coding problems.

I had to find a method for problem-solving—my career as a developer depended on it.

I immediately began researching methods. And I found one. In fact, what I uncovered was an invaluable strategy. It was a time-tested four-step method that was somehow under the radar in the developer ecosystem.

In this article, I’ll go over this four-step problem-solving method that you can use to start confidently solving coding problems.

Solving coding problems is not only part of the developer job interview process—it’s what a developer does all day. After all, writing code is problem-solving.

A method for solving problems

This method is from the book How to Solve It by George Pólya. It originally came out in 1945 and has sold over one million copies.

His problem-solving method has been used and taught by many programmers, from computer science professors (see Udacity’s Intro to CS course taught by professor David Evans) to modern web development teachers like Colt Steele.

Let’s walk through solving a simple coding problem using the four-step problem-solving method. This allows us to see the method in action as we learn it. We'll use JavaScript as our language of choice. Here’s the problem:

Create a function that adds together two numbers and returns that value. There are four steps to the problem-solving method:

• Understand the problem.
• Devise a plan.
• Carry out the plan.

Let’s get started with step one.

Step 1: Understand the problem.

When given a coding problem in an interview, it’s tempting to rush into coding. This is hard to avoid, especially if you have a time limit.

However, try to resist this urge. Make sure you actually understand the problem before you get started with solving it.

Read through the problem. If you’re in an interview, you could read through the problem out loud if that helps you slow down.

As you read through the problem, clarify any part of it you do not understand. If you’re in an interview, you can do this by asking your interviewer questions about the problem description. If you’re on your own, think through and/or Google parts of the question you might not understand.

This first step is vital as we often don’t take the time to fully understand the problem. When you don’t fully understand the problem, you’ll have a much harder time solving it.

To help you better understand the problem, ask yourself:

What are the inputs?

What kinds of inputs will go into this problem? In this example, the inputs are the arguments that our function will take.

Just from reading the problem description so far, we know that the inputs will be numbers. But to be more specific about what the inputs will be, we can ask:

Will the inputs always be just two numbers? What should happen if our function receives as input three numbers?

Here we could ask the interviewer for clarification, or look at the problem description further.

The coding problem might have a note saying, “You should only ever expect two inputs into the function.” If so, you know how to proceed. You can get more specific, as you’ll likely realize that you need to ask more questions on what kinds of inputs you might be receiving.

Will the inputs always be numbers? What should our function do if we receive the inputs “a” and “b”? Clarify whether or not our function will always take in numbers.

Optionally, you could write down possible inputs in a code comment to get a sense of what they’ll look like:

//inputs: 2, 4

What are the outputs?

What will this function return? In this case, the output will be one number that is the result of the two number inputs. Make sure you understand what your outputs will be.

Create some examples.

Once you have a grasp of the problem and know the possible inputs and outputs, you can start working on some concrete examples.

Examples can also be used as sanity checks to test your eventual problem. Most code challenge editors that you’ll work in (whether it’s in an interview or just using a site like Codewars or HackerRank) have examples or test cases already written for you. Even so, writing out your own examples can help you cement your understanding of the problem.

Start with a simple example or two of possible inputs and outputs. Let's return to our addition function.

Let’s call our function “add.”

What’s an example input? Example input might be:

// add(2, 3)

What is the output to this? To write the example output, we can write:

// add(2, 3) ---> 5

This indicates that our function will take in an input of 2 and 3 and return 5 as its output.

Create complex examples.

By walking through more complex examples, you can take the time to look for edge cases you might need to account for.

For example, what should we do if our inputs are strings instead of numbers? What if we have as input two strings, for example, add('a', 'b')?

Your interviewer might possibly tell you to return an error message if there are any inputs that are not numbers. If so, you can add a code comment to handle this case if it helps you remember you need to do this.

Your interviewer might also tell you to assume that your inputs will always be numbers, in which case you don’t need to write any extra code to handle this particular input edge case.

If you don’t have an interviewer and you’re just solving this problem, the problem might say what happens when you enter invalid inputs.

For example, some problems will say, “If there are zero inputs, return undefined.” For cases like this, you can optionally write a comment.

// check if there are no inputs.

// If no inputs, return undefined.

For our purposes, we’ll assume that our inputs will always be numbers. But generally, it’s good to think about edge cases.

Computer science professor Evans says to write what developers call defensive code. Think about what could go wrong and how your code could defend against possible errors.  

Before we move on to step 2, let’s summarize step 1, understand the problem:

-Read through the problem.

-What are the inputs?

-What are the outputs?

Create simple examples, then create more complex ones.

2. Devise a plan for solving the problem.

Next, devise a plan for how you’ll solve the problem. As you devise a plan, write it out in pseudocode.

Pseudocode is a plain language description of the steps in an algorithm. In other words, your pseudocode is your step-by-step plan for how to solve the problem.

Write out the steps you need to take to solve the problem. For a more complicated problem, you’d have more steps. For this problem, you could write:

// Create a sum variable.

Add the first input to the second input using the addition operator .

// Store value of both inputs into sum variable.

// Return as output the sum variable. Now you have your step-by-step plan to solve the problem. For more complex problems, professor Evans notes, “Consider systematically how a human solves the problem.” That is, forget about how your code might solve the problem for a moment, and think about how you would solve it as a human. This can help you see the steps more clearly.

3. Carry out the plan (Solve the problem!)

The next step in the problem-solving strategy is to solve the problem. Using your pseudocode as your guide, write out your actual code.

Professor Evans suggests focusing on a simple, mechanical solution. The easier and simpler your solution is, the more likely you can program it correctly.

Taking our pseudocode, we could now write this:

Professor Evans adds, remember not to prematurely optimize. That is, you might be tempted to start saying, “Wait, I’m doing this and it’s going to be inefficient code!”

First, just get out your simple, mechanical solution.

What if you can’t solve the entire problem? What if there's a part of it you still don't know how to solve?

Colt Steele gives great advice here: If you can’t solve part of the problem, ignore that hard part that’s tripping you up. Instead, focus on everything else that you can start writing.

Temporarily ignore that difficult part of the problem you don’t quite understand and write out the other parts. Once this is done, come back to the harder part.

This allows you to get at least some of the problem finished. And often, you’ll realize how to tackle that harder part of the problem once you come back to it.

Step 4: Look back over what you've done.

Once your solution is working, take the time to reflect on it and figure out how to make improvements. This might be the time you refactor your solution into a more efficient one.

As you look at your work, here are some questions Colt Steele suggests you ask yourself to figure out how you can improve your solution:

• Can you derive the result differently? What other approaches are there that are viable?
• Can you understand it at a glance? Does it make sense?
• Can you use the result or method for some other problem?
• Can you improve the performance of your solution?
• Can you think of other ways to refactor?
• How have other people solved this problem?

One way we might refactor our problem to make our code more concise: removing our variable and using an implicit return:

With step 4, your problem might never feel finished. Even great developers still write code that they later look at and want to change. These are guiding questions that can help you.

If you still have time in an interview, you can go through this step and make your solution better. If you are coding on your own, take the time to go over these steps.

When I’m practicing coding on my own, I almost always look at the solutions out there that are more elegant or effective than what I’ve come up with.

Wrapping Up

In this post, we’ve gone over the four-step problem-solving strategy for solving coding problems.

Let's review them here:

• Step 1: understand the problem.
• Step 2: create a step-by-step plan for how you’ll solve it .
• Step 3: carry out the plan and write the actual code.
• Step 4: look back and possibly refactor your solution if it could be better.

Practicing this problem-solving method has immensely helped me in my technical interviews and in my job as a developer. If you don't feel confident when it comes to solving coding problems, just remember that problem-solving is a skill that anyone can get better at with time and practice.

If you enjoyed this post, join my coding club , where we tackle coding challenges together every Sunday and support each other as we learn new technologies.

If you have feedback or questions on this post, feel free to tweet me @madisonkanna ..

Read more posts .

If you read this far, thank the author to show them you care. Say Thanks

Learn to code for free. freeCodeCamp's open source curriculum has helped more than 40,000 people get jobs as developers. Get started

Problem Solving

Foundations course, introduction.

Before we start digging into some pretty nifty JavaScript, we need to begin talking about problem solving : the most important skill a developer needs.

Problem solving is the core thing software developers do. The programming languages and tools they use are secondary to this fundamental skill.

From his book, “Think Like a Programmer” , V. Anton Spraul defines problem solving in programming as:

Problem solving is writing an original program that performs a particular set of tasks and meets all stated constraints.

The set of tasks can range from solving small coding exercises all the way up to building a social network site like Facebook or a search engine like Google. Each problem has its own set of constraints, for example, high performance and scalability may not matter too much in a coding exercise but it will be vital in apps like Google that need to service billions of search queries each day.

New programmers often find problem solving the hardest skill to build. It’s not uncommon for budding programmers to breeze through learning syntax and programming concepts, yet when trying to code something on their own, they find themselves staring blankly at their text editor not knowing where to start.

The best way to improve your problem solving ability is by building experience by making lots and lots of programs. The more practice you have the better you’ll be prepared to solve real world problems.

In this lesson we will walk through a few techniques that can be used to help with the problem solving process.

Lesson overview

This section contains a general overview of topics that you will learn in this lesson.

• Explain the three steps in the problem solving process.
• Explain what pseudocode is and be able to use it to solve problems.
• Be able to break a problem down into subproblems.

Understand the problem

The first step to solving a problem is understanding exactly what the problem is. If you don’t understand the problem, you won’t know when you’ve successfully solved it and may waste a lot of time on a wrong solution .

To gain clarity and understanding of the problem, write it down on paper, reword it in plain English until it makes sense to you, and draw diagrams if that helps. When you can explain the problem to someone else in plain English, you understand it.

Now that you know what you’re aiming to solve, don’t jump into coding just yet. It’s time to plan out how you’re going to solve it first. Some of the questions you should answer at this stage of the process:

• Does your program have a user interface? What will it look like? What functionality will the interface have? Sketch this out on paper.
• What inputs will your program have? Will the user enter data or will you get input from somewhere else?
• What’s the desired output?
• Given your inputs, what are the steps necessary to return the desired output?

The last question is where you will write out an algorithm to solve the problem. You can think of an algorithm as a recipe for solving a particular problem. It defines the steps that need to be taken by the computer to solve a problem in pseudocode.

Pseudocode is writing out the logic for your program in natural language instead of code. It helps you slow down and think through the steps your program will have to go through to solve the problem.

Here’s an example of what the pseudocode for a program that prints all numbers up to an inputted number might look like:

This is a basic program to demonstrate how pseudocode looks. There will be more examples of pseudocode included in the assignments.

Divide and conquer

From your planning, you should have identified some subproblems of the big problem you’re solving. Each of the steps in the algorithm we wrote out in the last section are subproblems. Pick the smallest or simplest one and start there with coding.

It’s important to remember that you might not know all the steps that you might need up front, so your algorithm may be incomplete -— this is fine. Getting started with and solving one of the subproblems you have identified in the planning stage often reveals the next subproblem you can work on. Or, if you already know the next subproblem, it’s often simpler with the first subproblem solved.

Many beginners try to solve the big problem in one go. Don’t do this . If the problem is sufficiently complex, you’ll get yourself tied in knots and make life a lot harder for yourself. Decomposing problems into smaller and easier to solve subproblems is a much better approach. Decomposition is the main way to deal with complexity, making problems easier and more approachable to solve and understand.

In short, break the big problem down and solve each of the smaller problems until you’ve solved the big problem.

Solving Fizz Buzz

To demonstrate this workflow in action, let’s solve a common programming exercise: Fizz Buzz, explained in this wiki article .

Understanding the problem

Write a program that takes a user’s input and prints the numbers from one to the number the user entered. However, for multiples of three print Fizz instead of the number and for the multiples of five print Buzz . For numbers which are multiples of both three and five print FizzBuzz .

This is the big picture problem we will be solving. But we can always make it clearer by rewording it.

Write a program that allows the user to enter a number, print each number between one and the number the user entered, but for numbers that divide by 3 without a remainder print Fizz instead. For numbers that divide by 5 without a remainder print Buzz and finally for numbers that divide by both 3 and 5 without a remainder print FizzBuzz .

Does your program have an interface? What will it look like? Our FizzBuzz solution will be a browser console program, so we don’t need an interface. The only user interaction will be allowing users to enter a number.

What inputs will your program have? Will the user enter data or will you get input from somewhere else? The user will enter a number from a prompt (popup box).

What’s the desired output? The desired output is a list of numbers from 1 to the number the user entered. But each number that is divisible by 3 will output Fizz , each number that is divisible by 5 will output Buzz and each number that is divisible by both 3 and 5 will output FizzBuzz .

Writing the pseudocode

What are the steps necessary to return the desired output? Here is an algorithm in pseudocode for this problem:

Dividing and conquering

As we can see from the algorithm we developed, the first subproblem we can solve is getting input from the user. So let’s start there and verify it works by printing the entered number.

With JavaScript, we’ll use the “prompt” method.

The above code should create a little popup box that asks the user for a number. The input we get back will be stored in our variable answer .

We wrapped the prompt call in a parseInt function so that a number is returned from the user’s input.

With that done, let’s move on to the next subproblem: “Loop from 1 to the entered number”. There are many ways to do this in JavaScript. One of the common ways - that you actually see in many other languages like Java, C++, and Ruby - is with the for loop :

If you haven’t seen this before and it looks strange, it’s actually straightforward. We declare a variable i and assign it 1: the initial value of the variable i in our loop. The second clause, i <= answer is our condition. We want to loop until i is greater than answer . The third clause, i++ , tells our loop to increment i by 1 every iteration. As a result, if the user inputs 10, this loop would print numbers 1 - 10 to the console.

Most of the time, programmers find themselves looping from 0. Due to the needs of our program, we’re starting from 1

With that working, let’s move on to the next problem: If the current number is divisible by 3, then print Fizz .

We are using the modulus operator ( % ) here to divide the current number by three. If you recall from a previous lesson, the modulus operator returns the remainder of a division. So if a remainder of 0 is returned from the division, it means the current number is divisible by 3.

After this change the program will now output this when you run it and the user inputs 10:

The program is starting to take shape. The final few subproblems should be easy to solve as the basic structure is in place and they are just different variations of the condition we’ve already got in place. Let’s tackle the next one: If the current number is divisible by 5 then print Buzz .

When you run the program now, you should see this output if the user inputs 10:

We have one more subproblem to solve to complete the program: If the current number is divisible by 3 and 5 then print FizzBuzz .

We’ve had to move the conditionals around a little to get it to work. The first condition now checks if i is divisible by 3 and 5 instead of checking if i is just divisible by 3. We’ve had to do this because if we kept it the way it was, it would run the first condition if (i % 3 === 0) , so that if i was divisible by 3, it would print Fizz and then move on to the next number in the iteration, even if i was divisible by 5 as well.

With the condition if (i % 3 === 0 && i % 5 === 0) coming first, we check that i is divisible by both 3 and 5 before moving on to check if it is divisible by 3 or 5 individually in the else if conditions.

The program is now complete! If you run it now you should get this output when the user inputs 20:

• Read How to Think Like a Programmer - Lessons in Problem Solving by Richard Reis.
• Watch How to Begin Thinking Like a Programmer by Coding Tech. It’s an hour long but packed full of information and definitely worth your time watching.
• Read this Pseudocode: What It Is and How to Write It article from Built In.

Knowledge check

This section contains questions for you to check your understanding of this lesson on your own. If you’re having trouble answering a question, click it and review the material it links to.

• What are the three stages in the problem solving process?
• Why is it important to clearly understand the problem first?
• What can you do to help get a clearer understanding of the problem?
• What are some of the things you should do in the planning stage of the problem solving process?
• What is an algorithm?
• What is pseudocode?
• What are the advantages of breaking a problem down and solving the smaller problems?

Additional resources

This section contains helpful links to other content. It isn’t required, so consider it supplemental.

• Read the first chapter in Think Like a Programmer: An Introduction to Creative Problem Solving ( not free ). This book’s examples are in C++, but you will understand everything since the main idea of the book is to teach programmers to better solve problems. It’s an amazing book and worth every penny. It will make you a better programmer.
• Watch this video on repetitive programming techniques .
• Watch Jonathan Blow on solving hard problems where he gives sage advice on how to approach problem solving in software projects.

Support us!

The odin project is funded by the community. join us in empowering learners around the globe by supporting the odin project.

Tutorial Playlist

Programming tutorial, your guide to the best backend languages for 2024, an ultimate guide that helps you to start learn coding 2024, what is backend development: the ultimate guide for beginners, all you need to know for choosing the first programming language to learn, here’s all you need to know about coding, decoding, and reasoning with examples, understanding what is xml: the best guide to xml and its concepts., an ultimate guide to learn the importance of low-code and no-code development, top frontend languages that you should know about, top 75+ frontend developer interview questions and answers, the ultimate guide to learn typescript generics, the most comprehensive guide for beginners to know ‘what is typescript’.

The Ultimate Guide on Introduction to Competitive Programming

Top 60+ TCS NQT Interview Questions and Answers for 2024

Most commonly asked logical reasoning questions in an aptitude test, everything you need to know about advanced typescript concepts, an absolute guide to build c hello world program, a one-stop solution guide to learn how to create a game in unity, what is nat significance of nat for translating ip addresses in the network model, data science vs software engineering: key differences, a real-time chat application typescript project using node.js as a server, what is raspberry pi here’s the best guide to get started, what is arduino here’s the best beginners guide to get started, arduino vs. raspberry pi: which is the better board, the perfect guide for all you need to learn about mean stack, software developer resume: a comprehensive guide, here’s everything all you need to know about the programming roadmap, an ultimate guide that helps you to develop and improve problem solving in programming, the top 10 awesome arduino projects of all time, roles of product managers, pyspark rdd: everything you need to know about pyspark rdd, wipro interview questions and answers that you should know before going for an interview, how to use typescript with nodejs: the ultimate guide, what is rust programming language why is it so popular, software terminologies, an ultimate guide that helps you to develop and improve problem solving in programming.

Lesson 27 of 34 By Hemant Deshpande

Table of Contents

Coding and Programming skills hold a significant and critical role in implementing and developing various technologies and software. They add more value to the future and development. These programming and coding skills are essential for every person to improve problem solving skills. So, we brought you this article to help you learn and know the importance of these skills in the future. 

Want a Top Software Development Job? Start Here!

Topics covered in this problem solving in programming article are:

• What is Problem Solving in Programming? 
• Problem Solving skills in Programming
• How does it impact your career ?
• Steps involved in Problem Solving
• Steps to improve Problem Solving in programming

What is Problem Solving in Programming?

Computers are used to solve various problems in day-to-day life. Problem Solving is an essential skill that helps to solve problems in programming. There are specific steps to be carried out to solve problems in computer programming, and the success depends on how correctly and precisely we define a problem. This involves designing, identifying and implementing problems using certain steps to develop a computer.

When we know what exactly problem solving in programming is, let us learn how it impacts your career growth.

How Does It Impact Your Career?

Many companies look for candidates with excellent problem solving skills. These skills help people manage the work and make candidates put more effort into the work, which results in finding solutions for complex problems in unexpected situations. These skills also help to identify quick solutions when they arise and are identified. 

People with great problem solving skills also possess more thinking and analytical skills, which makes them much more successful and confident in their career and able to work in any kind of environment. 

The above section gives you an idea of how problem solving in programming impacts your career and growth. Now, let's understand what problem solving skills mean.

Problem Solving Skills in Programming

Solving a question that is related to computers is more complicated than finding the solutions for other questions. It requires excellent knowledge and much thinking power. Problem solving in programming skills is much needed for a person and holds a major advantage. For every question, there are specific steps to be followed to get a perfect solution. By using those steps, it is possible to find a solution quickly.

The above section is covered with an explanation of problem solving in programming skills. Now let's learn some steps involved in problem solving.

Steps Involved in Problem Solving

Before being ready to solve a problem, there are some steps and procedures to be followed to find the solution. Let's have a look at them in this problem solving in programming article.

Basically, they are divided into four categories:

• Analysing the problem
• Developing the algorithm
• Testing and debugging

Analysing the Problem

Every problem has a perfect solution; before we are ready to solve a problem, we must look over the question and understand it. When we know the question, it is easy to find the solution for it. If we are not ready with what we have to solve, then we end up with the question and cannot find the answer as expected. By analysing it, we can figure out the outputs and inputs to be carried out. Thus, when we analyse and are ready with the list, it is easy and helps us find the solution easily. 

Developing the Algorithm

It is required to decide a solution before writing a program. The procedure of representing the solution  in a natural language called an algorithm. We must design, develop and decide the final approach after a number of trials and errors, before actually writing the final code on an algorithm before we write the code. It captures and refines all the aspects of the desired solution.

Once we finalise the algorithm, we must convert the decided algorithm into a code or program using a dedicated programming language that is understandable by the computer to find a desired solution. In this stage, a wide variety of programming languages are used to convert the algorithm into code. 

Testing and Debugging

The designed and developed program undergoes several rigorous tests based on various real-time parameters and the program undergoes various levels of simulations. It must meet the user's requirements, which have to respond with the required time. It should generate all expected outputs to all the possible inputs. The program should also undergo bug fixing and all possible exception handling. If it fails to show the possible results, it should be checked for logical errors.

Industries follow some testing methods like system testing, component testing and acceptance testing while developing complex applications. The errors identified while testing are debugged or rectified and tested again until all errors are removed from the program.

The steps mentioned above are involved in problem solving in programming. Now let's see some more detailed information about the steps to improve problem solving in programming.

Steps to Improve Problem Solving in Programming

Right mindset.

The way to approach problems is the key to improving the skills. To find a solution, a positive mindset helps to solve problems quickly. If you think something is impossible, then it is hard to achieve. When you feel free and focus with a positive attitude, even complex problems will have a perfect solution.

Making Right Decisions

When we need to solve a problem, we must be clear with the solution. The perfect solution helps to get success in a shorter period. Making the right decisions in the right situation helps to find the perfect solution quickly and efficiently. These skills also help to get more command over the subject.

Keeping Ideas on Track

Ideas always help much in improving the skills; they also help to gain more knowledge and more command over things. In problem solving situations, these ideas help much and help to develop more skills. Give opportunities for the mind and keep on noting the ideas.

Learning from Feedbacks

A crucial part of learning is from the feedback. Mistakes help you to gain more knowledge and have much growth. When you have a solution for a problem, go for the feedback from the experienced or the professionals. It helps you get success within a shorter period and enables you to find other solutions easily.

Asking Questions

Questions are an incredible part of life. While searching for solutions, there are a lot of questions that arise in our minds. Once you know the question correctly, then you are able to find answers quickly. In coding or programming, we must have a clear idea about the problem. Then, you can find the perfect solution for it. Raising questions can help to understand the problem.

These are a few reasons and tips to improve problem solving in programming skills. Now let's see some major benefits in this article.

• Problem solving in programming skills helps to gain more knowledge over coding and programming, which is a major benefit.
• These problem solving skills also help to develop more skills in a person and build a promising career.
• These skills also help to find the solutions for critical and complex problems in a perfect way.
• Learning and developing problem solving in programming helps in building a good foundation.
• Most of the companies are looking for people with good problem solving skills, and these play an important role when it comes to job opportunities 

Don't miss out on the opportunity to become a Certified Professional with Simplilearn's Post Graduate Program in Full Stack Web Development . Enroll Today!

Problem solving in programming skills is important in this modern world; these skills build a great career and hold a great advantage. This article on problem solving in programming provides you with an idea of how it plays a massive role in the present world. In this problem solving in programming article, the skills and the ways to improve more command on problem solving in programming are mentioned and explained in a proper way.

If you are looking to advance in your career. Simplilearn provides training and certification courses on various programming languages - Python , Java , Javascript , and many more. Check out our Post Graduate Program in Full Stack Web Development course that will help you excel in your career.

If you have any questions for us on the problem solving in programming article. Do let us know in the comments section below; we have our experts answer it right away.

Find our Full Stack Developer - MERN Stack Online Bootcamp in top cities:

About the author.

Hemant Deshpande, PMP has more than 17 years of experience working for various global MNC's. He has more than 10 years of experience in managing large transformation programs for Fortune 500 clients across verticals such as Banking, Finance, Insurance, Healthcare, Telecom and others. During his career he has worked across the geographies - North America, Europe, Middle East, and Asia Pacific. Hemant is an internationally Certified Executive Coach (CCA/ICF Approved) working with corporate leaders. He also provides Management Consulting and Training services. He is passionate about writing and regularly blogs and writes content for top websites. His motto in life - Making a positive difference.

Recommended Resources

Your One-Stop Solution to Understand Coin Change Problem

Combating the Global Talent Shortage Through Skill Development Programs

What Is Problem Solving? Steps, Techniques, and Best Practices Explained

One Stop Solution to All the Dynamic Programming Problems

The Ultimate Guide to Top Front End and Back End Programming Languages for 2021

• PMP, PMI, PMBOK, CAPM, PgMP, PfMP, ACP, PBA, RMP, SP, and OPM3 are registered marks of the Project Management Institute, Inc.

• Latest Articles
• Top Articles
• Posting/Update Guidelines
• Article Help Forum

• View Unanswered Questions
• View All Questions
• View C# questions
• View C++ questions
• View Javascript questions
• View Visual Basic questions
• View Python questions
• CodeProject.AI Server
• All Message Boards...
• Running a Business
• Sales / Marketing
• Collaboration / Beta Testing
• Work Issues
• Design and Architecture
• Artificial Intelligence
• Internet of Things
• ATL / WTL / STL
• Managed C++/CLI
• Objective-C and Swift
• System Admin
• Hosting and Servers
• Linux Programming
• .NET (Core and Framework)
• Visual Basic
• Web Development
• Site Bugs / Suggestions
• Spam and Abuse Watch
• Competitions
• The Insider Newsletter
• The Daily Build Newsletter
• Newsletter archive
• CodeProject Stuff
• Most Valuable Professionals
• The Lounge  
• The CodeProject Blog
• Where I Am: Member Photos
• The Insider News
• The Weird & The Wonderful
• What is 'CodeProject'?
• General FAQ
• Ask a Question
• Bugs and Suggestions

The Beginner Programmer's guide to Problem Solving [With Example]

Have you got this feeling that you are able to grasp the concepts of programming and  you are able to understand what’s a variable, what’s a function, what are data types, etc. yet you find it difficult to solve problems in programming.  Every beginner gets this feeling.  I did too when starting out.

It is important to overcome this feeling at the earliest, otherwise it can form a mental block for you.

How it can be a mental block to you?  Common sense says that the more you practice a certain skill, you get better at that skill as time progresses.  Same goes with problem solving too.  The more problems you solve, the better you become at problem solving.  But when you get a feel that you are trying hard and still unable to solve a problem or find it extremely difficult, your confidence lowers.  At this stage, either you stop solving problems or try to solve lesser number of problems.

The point is your curriculum or your professional work is generally designed in such a manner that the order of difficulty increases as time progresses.  So, you are in a situation where you feel less confident in solving small problems but now tasked with solving bigger problems.  And the cycle continues till it becomes a permanent mental block in you.

Is it too late to start solving problems?

No.  If you have come to the realization that you need to improve your problem solving skills, you have made that good first step.  Quite often our egos don’t let us accept the obvious.  It is good to accept certain truth because that is the only way that we can improve ourselves.

What can I do to become better at solving problems?

Remove the mental block first – exercise your mind.

Your mind is your most powerful weapon.  So you have to think you can actually solve the problem.  So from today, think positively that you can solve any problem.  But you will obviously start with small problems and go on to solve bigger problems.

As with every aspect in life, it starts with conditioning the mind.  So, starting today, tell yourselves the following:

• I can solve any problem that is put at me
• I will commit at least 1-2 hours per day on solving problems alone for the next 30 days
• I will never give up on any problem that is put at me, I will ask for help if required.1

Understand the basic approach to problem solving

Do you know one of the reasons for your struggle with problem solving?  One reason might be due to lack of practice.  But the main reason is because you have not understood the basics of problem solving especially in programming.  Once you understand the approach to problem solving to the smallest of things, you can go ahead and solve bigger and more complex problems with confidence.1

Ever wondered how top tech companies like Google, Amazon solved the internet’s biggest & hardest problems?  The answer is simplicity.  They solved problems at the basic level and then went on to solve bigger and bigger problems.  You can do it too.  But you need to be good at the basics.

What do I need to understand before even trying to solve the problem?

Understand the problem clearly – the power of clarity.

You need to understand your problem clearly before even trying to solve it1.  Lack of clarity at this stage will put you down.  So make a conscious effort in understanding the problem more clearly.  Ask questions like What, Why, When, Where, What if and How.  Not all questions might be applicable to your problem, but it is important to ask questions to yourself at this stage before you go ahead trying to solve the problem.

Visualize – The Power of visualization

I am sure everyone of you is aware of what visualization is.  Trying to picturize your thoughts.  Have you ever imagined how some people can solve extra ordinary problems just by looking into those problems and they will instantly have a solution to it?  And we don’t even understand the problem fully?  It is because they do it with their mind.  They visualize the problem in their minds and they solve it in their minds itself.  Visualization is a powerful tool in your mind.

But in order to get to that state, first you need to visualize the problem externally.  That is where a pen and a paper/notebook (or) a white board comes into play1.  Try to visualize the problem at hand and try to picturize the problem.  That is also one of the steps to make sure that you understand the problem clearly.

There was a situation when I and my dear friend & colleague were discussing about a problem and we were literally going nowhere.  This was actually when we each had around 7 years of experience in the industry.  At that point, my friend said “Let’s put our points in board.  If we don’t put it on the board, we will never get started”.  And we started putting things on board.  Things started to get more clear and raised more questions and ultimately became more clear.

That is the power of visualization.  It really helps us to get started with our thinking. This visual thing works.  Just try it out.

Your next question might be “I kinda get it, but I don’t.  How do I visualize? What exactly do I visualize?”.  Please read on to find out the answers.

What is the basic approach to problem solving

Step 1:  identify small problems.

The major trick in problem solving is to identify and solve the smallest problem and then moving ahead with bigger ones.  So how do you do it?

The answer is division of responsibility.  Simply put, we need to identify parts that can stand on its own and identify a sequence in those responsibilities.  And once you start breaking down the problems into smaller ones, then you can go ahead with the next step.

Step 2:  Solve the smaller problems one at a time

Now that you have identified the smaller problems, try to solve them.  While solving them, make sure that you are focussing only on one problem at a time.  That makes life much simpler for us.  If you feel that this smaller problem is too big to solve on its own, try to break it down further.  You need to iterate steps 1 to step 3 for each smaller problem.  But for now, ignore the bigger problem and solve the rest of the problems.

• It is ok to assume that other problems are solved
• It is ok to hardcode when coding a particular problem, but later you will resolve it in step 3.
• Solve the easier problems first, that will give you confidence and momentum until you get the confidence to solve the hardest problem first.

Step 3: Connect the dots (Integration)

You have solved individual problems.  Now it is time to connect the dots by connecting the individual solution.  Identify those steps which will make the solution or the program complete.  Typically in programming, the dots are connected by passing data that is stored in variables.

Step 4: Try to optimize each step & across steps

Once you are completed with a working solution, try to optimize the solution with the best code that you can write.  This comes only with practice.  This trick can make a difference between a good programmer and a great programmer.  But to get to this step, you need to be first good at steps 1 to 3.

Let’s take an example & walkthrough the problem solving approach

Problem:  check if a user given string is a palindrome or not.

I will be using Python for this exercise (Although I have experience in1 C# and JAVA, I am also a Python beginner, so pardon any bad code).  Let’s iterate through our steps:

Let’s call this as Level 1:

Step 1:  Identify smaller problems:

Step 2: Solve the small problems

So each small problem will map to its corresponding solution as below:

Note: When solving the step (3.  Compare the variables), I am doing 2 things:

• I am making an assumption that reversed is the variable name of the reversed string.
• I am hardcoding the variable name reversed to ‘madam’ to avoid compile time error
• If you execute the program at this state, you can input ‘madam’ and check if it is printing ‘The given string is a palindrome’ (And) you can input something else like ‘dog’ and check if it is printing ‘The given string is not a palindrome’

When we are trying to connect the dots, the only thing that is missing now is the variable reversed is hardcoded.  For that to be set to the correct value, we need to break the small problem (Reverse the user input and store in a separate variable) into further smaller problems.  Till that point we need to mark it as incomplete.

2 things still remain unsolved in Level 1:

• Solution for step 2 in the diagram (Reverse the user input and store in a separate variable)
• Connecting the dots once the solution for step 2 is found

Iterating small problem 2 through our problem solving steps:

Let’s call this Level 2:

Step 1: Identify smaller problems

Step 3: Connect the dots

Here, we have already connected the dots.  So we need not do anything extra in this step.

Now we have solved the smaller problems, which means Level 2 is over.  Now we need to come back to Level 1.

If you remember, 2 things remain in Level 1.  One is solution for step 2 which we have found now.  Two is connecting the dots.

Now if we substitute the small problem 2 with the solution that we derived just now, we get something like this:

The thing that remains is connecting the dots.

So if we see what is the missing connection, the variable reversed is set twice.  One to the solution of step 2 and another is hardcoded in step 3.  So we can now remove the hardcoded value in step 3, in which case our code will become like this

If you see, we have actually solved our problem.

We are left with step 4 – Optimize each step and across steps

Step 4: Try to optimize each step and across steps

As you can see, there are many things that needs to be optimized for this code.  I would leave you to optimize the code further.  Come on, put on your thinking cap and try different solutions.

BONUS STEP 5:  Make the code robust

By robust I mean,

• Adding error & exception handling
• Using better variable names
• Adding user defined functions
• Adding comments where necessary

Again, I would leave you to figure out how to do this step.

• We saw just how we can solve problems using a step by step approach
• By solving smaller problems, I get into a momentum for solving bigger & tougher problems
• By focussing one problem at a time, I am eliminating distractions, thus allowing to better direct your efforts for that one problem rather than getting confused with many small problems at hand.
• If you understand this approach and practice, you will definitely go on to solve bigger problems and your confidence will raise.
• Beauty about breaking down the problem is that we can further convert each problem and sub problem into separate functions/modules thus making the code more modularized and maintainable.

Wait, You can’t leave yet:

Now dear beginner programmers, take any problem and try to apply this approach.  See the results for yourselves.  Now, describe the following in the comments section:

• What problem you are solving?
• How did you break it down? (Even a snap of your notebook page or board will do!)
• The final code
• How did you feel and what did you learn from this exercise?

Also remember, I am challenging you for the 30 day problem solving challenge.

If you liked this blog post, please feel free to share it with your circles in social media.

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Comments and Discussions

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Member-only story

10 Steps to Solving a Programming Problem

Tips for new developers staring at a blank screen, unsure of where to start.

Valinda Chan

Some of the feedback I hear from new developers working on a programming problem revolves around uncertainty of where to start. You understand the problem, the logic, basics of the syntax, etc. If you see someone else’s code or have someone to guide you, you can follow along. But maybe you feel uncertain about doing it yourself and have trouble turning your thoughts into code at first even though you understand the syntax or logic. Here’s my process and some tips to tackling a sample problem that hopefully some of you may find helpful in your journey.

1. Read the problem at least three times (or however many makes you feel comfortable)

You can’t solve a problem you don’t understand. There is a difference between the problem and the problem you think you are solving. It’s easy to start reading the first few lines in a problem and assume the rest of it because it’s similar to something you’ve seen in the past. If you are making even a popular game like Hangman, be sure to read through any rules even if you’ve played it before. I once was asked to make a game like Hangman that I realized was “Evil Hangman” only after I read through the instructions (it was a trick!).

Sometimes I’ll even try explaining the problem to a friend and see if her understanding of my explanation matches the problem I am tasked with. You don’t want to find out halfway through that you misunderstood the problem. Taking extra time in the beginning is worth it. The better you understand the problem, the easier it will be to solve it.

Let’s pretend we are creating a simple function selectEvenNumbers that will take in an array of numbers and return an array evenNumbers of only even numbers. If there are no even numbers, return the empty array evenNumbers .

Here are some questions that run through my mind:

• How can a computer tell what is an even number? Divide that number by 2 and see if its remainder is 0.
• What am I passing into this function? An array
• What will that array contain? One or more numbers
• What are the data types of the elements in the array? Numbers
• What is the goal of this function? What am I returning at the end of this function? The goal is to take all the even numbers and return them in an array. If there are no even numbers, return an empty array.

2. Work through the problem manually with at least three sets of sample data

Take out a piece of paper and work through the problem manually. Think of at least three sets of sample data you can use. Consider corner and edge cases as well.

Corner case : a problem or situation that occurs outside of normal operating parameters, specifically when multiple environmental variables or conditions are simultaneously at extreme levels, even though each parameter is within the specified range for that parameter. Edge case : problem or situation that occurs only at an extreme (maximum or minimum) operating parameter

For example, below are some sets of sample data to use:

When you are first starting out, it is easy to gloss over the steps. Because your brain may already be familiar with even numbers, you may just look at a sample set of data and pull out numbers like 2 , 4 , 6 and so forth in the array without fully being aware of each and every step your brain is taking to solve it. If this is challenging, try using large sets of data as it will override your brain’s ability to naturally solve the problem just by looking at it. That helps you work through the real algorithm.

Let’s go through the first array [1]

• Look at the only element in the array [1]
• Decide if it is even. It is not
• Notice that there are no more elements in this array
• Determine there are no even numbers in this provided array
• Return an empty array

Let’s go through the array [1, 2]

• Look at the first element in array [1, 2]
• Look at the next element in the array
• Decide if it is even. It is even
• Make an array evenNumbers and add 2 to this array
• Return the array evenNumbers which is [2]

I go through this a few more times. Notice how the steps I wrote down for [1] varies slightly from [1, 2] . That is why I try to go through a couple of different sets. I have some sets with just one element, some with floats instead of just integers, some with multiple digits in an element, and some with negatives just to be safe.

3. Simplify and optimize your steps

Look for patterns and see if there’s anything you can generalize. See if you can reduce any steps or if you are repeating any steps.

• Create a function selectEvenNumbers
• Create a new empty array evenNumbers where I store even numbers, if any
• Go through each element in the array [1, 2]
• Find the first element
• Decide if it is even by seeing if it is divisible by 2. If it is even, I add that to evenNumbers
• Find the next element
• Repeat step #4
• Repeat step #5 and #4 until there are no more elements in this array
• Return the array evenNumbers , regardless of whether it has anything in it

This approach may remind you of Mathematical Induction in that you:

• Show it is true for n = 1 , n = 2 , ...
• Suppose it is true for n = k
• Prove it is true for n = k + 1

4. Write pseudocode

Even after you’ve worked out general steps, writing out pseudocode that you can translate into code will help with defining the structure of your code and make coding a lot easier. Write pseudocode line by line. You can do this either on paper or as comments in your code editor. If you’re starting out and find blank screens to be daunting or distracting, I recommend doing it on paper.

Pseudocode generally does not actually have specific rules in particular but sometimes, I might end up including some syntax from a language just because I am familiar enough with an aspect of the programming language. Don’t get caught up with the syntax. Focus on the logic and steps.

For our problem, there are many different ways to do this. For example, you can use filter but for the sake of keeping this example as easy to follow along as possible, we will use a basic for loop for now (but we will use filter later when we refactor our code).

Here is an example of pseudocode that has more words:

Here is an example of pseudocode that has fewer words:

Either way is fine as long as you are writing it out line-by-line and understand the logic on each line.

Refer back to the problem to make sure you are on track.

5. Translate pseudocode into code and debug

When you have your pseudocode ready, translate each line into real code in the language you are working on. We will use JavaScript for this example.

If you wrote it out on paper, type this up as comments in your code editor. Then replace each line in your pseudocode.

Then I call the function and give it some sample sets of data we used earlier. I use them to see if my code returns the results I want. You can also write tests to check if the actual output is equal to the expected output.

I generally use console.log() after each variable or line or so. This helps me check if the values and code are behaving as expected before I move on . By doing this, I catch any issues before I get too far. Below is an example of what values I would check when I am first starting out. I do this throughout my code as I type it out.

After working though each line of my pseudocode, below is what we end up with. // is what the line was in pseudocode. Text that is bolded is the actual code in JavaScript.

I get rid of the pseudocode to avoid confusion.

Sometimes new developers will get hung up with the syntax that it becomes difficult to move forward. Remember that syntax will come more naturally over time and there is no shame in referencing material for the correct syntax later on when coding.

6. Simplify and optimize your code

You’ve probably noticed by now that simplifying and optimizing are recurring themes.

“Simplicity is prerequisite for reliability.” — Edsger W. Dijkstra, Dutch computer scientist and early pioneer in many research areas of computing science

In this example, one way of optimizing it would be to filter out items from an array by returning a new array using filter . This way, we don’t have to define another variable evenNumbers because filter will return a new array with copies of elements that match the filter. This will not change the original array. We also don’t need to use a for loop with this approach. filter will go through each item, return either true , to have that element in the array, or false to skip it.

Simplifying and optimizing your code may require you to iterate a few times, identifying ways to further simplify and optimize code.

Here are some questions to keep in mind:

• What are your goals for simplifying and optimizing? The goals will depend on your team’s style or your personal preference. Are you trying to condense the code as much as possible? Is the goal to make it the code more readable? If that’s the case, you may prefer taking that extra line to define the variable or compute something rather than trying to define and compute all in one line.
• How else can you make the code more readable?
• Are there any more extra steps you can take out?
• Are there any variables or functions you ended up not even needing or using?
• Are you repeating some steps a lot? See if you can define in another function.
• Are there better ways to handle edge cases?

“Programs must be written for people to read, and only incidentally for machines to execute.” — Gerald Jay Sussman and Hal Abelson, Authors of “Structure and Interpretation of Computer Programs”

This step really should be throughout the process. Debugging throughout will help you catch any syntax errors or gaps in logic sooner rather than later. Take advantage of your Integrated Development Environment (IDE) and debugger. When I encounter bugs, I trace the code line-by-line to see if there was anything that did not go as expected. Here are some techniques I use:

• Check the console to see what the error message says. Sometimes it’ll point out a line number I need to check. This gives me a rough idea of where to start, although the issue sometimes may not be at this line at all.
• Comment out chunks or lines of code and output what I have so far to quickly see if the code is behaving how I expected. I can always uncomment the code as needed.
• Use other sample data if there are scenarios I did not think of and see if the code will still work.
• Save different versions of my file if I am trying out a completely different approach. I don’t want to lose any of my work if I end up wanting to revert back to it!

“The most effective debugging tool is still careful thought, coupled with judiciously placed print statements.” — Brian W. Kernighan, Computer Science Professor at Princeton University

8. Write useful comments

You may not always remember what every single line meant a month later. And someone else working on your code may not know either. That’s why it’s important to write useful comments to avoid problems and save time later on if you need to come back to it.

Stay away from comments such as:

// This is an array. Iterate through it.

// This is a variable

I try to write brief, high-level comments that help me understand what’s going on if it is not obvious. This comes in handy when I am working on more complex problems. It helps understand what a particular function is doing and why. Through the use of clear variable names, function names, and comments, you (and others) should be able to understand:

• What is this code for?
• What is it doing?

9. Get feedback through code reviews

Get feedback from your teammates, professors, and other developers. Check out Stack Overflow . See how others tackled the problem and learn from them. There are sometimes several ways to approach a problem. Find out what they are and you’ll get better and quicker at coming up with them yourself.

“No matter how slow you are writing clean code, you will always be slower if you make a mess.” — Uncle Bob Martin, Software Engineer and Co-author of the Agile Manifesto

10. Practice, practice, practice

Even experienced developers are always practicing and learning. If you get helpful feedback, implement it. Redo a problem or do similar problems. Keep pushing yourself. With each problem you solve, the better a developer you become. Celebrate each success and be sure to remember how far you’ve come. Remember that programming, like with anything, comes easier and more naturally with time.

“Take pride in how far you’ve come. Have faith in how far you can go. But don’t forget to enjoy the journey.” — Michael Josephson, Founder of Joseph and Edna Josephson Institute of Ethics

Thanks Gavin Stark

Written by Valinda Chan

Product & UX Design

More from Valinda Chan and codeburst

Getting it right: why infographics are not the same as data visualizations

A rundown on the key differences and ideas for when to use each.

Top 50 Java Interview Questions for Beginners and Junior Developers

A list of frequently asked java questions and answers from programming job interviews of java developers of different experience..

How To Create Horizontal Scrolling Containers

As a front end developer, more and more frequently i am given designs that include a horizontal scrolling component. this has become….

ThinkGrowth.org

Leading with the Power of Influence vs. Positional Power

Getting people to take risks in a risk-averse environment, recommended from medium.

Benoit Ruiz

Better Programming

Advice From a Software Engineer With 8 Years of Experience

Practical tips for those who want to advance in their careers.

Benedict Neo

bitgrit Data Science Publication

Roadmap to Learn AI in 2024

A free curriculum for hackers and programmers to learn ai.

General Coding Knowledge

Stories to Help You Grow as a Software Developer

Coding & Development

ChatGPT prompts

Hayk Simonyan

Level Up Coding

System Design Interview Question: Design Spotify

High-level overview of a system design interview question - design spotify..

Artturi Jalli

I Built an App in 6 Hours that Makes $1,500/Mo

Copy my strategy.

Vaishnav Manoj

DataX Journal

JSON is incredibly slow: Here’s What’s Faster!

Unlocking the need for speed: optimizing json performance for lightning-fast apps and finding alternatives to it.

Gowtham Oleti

Apps I Use And Why You Should Too.

Let’s skip past the usual suspects like youtube, whatsapp and instagram. i want to share with you some less familiar apps that have become….

Text to speech

• Streaming and Technology

What Is Problem-Solving Programming?

Are you interested in a career in computer programming? Are you wondering if you have what it takes? Perhaps you’re fluent in relevant programming languages, but you wonder what other skills are essential to a successful programming career.

Here, we’ll look at how problem-solving skills are essential for programmers. This article will equip you to leverage your problem-solving skills on your programming resume.  

A Career in Programming

What is programming? We’re not talking about television programming here. Programmers are also known as computer programmers, systems programmers, software engineers, or coders. Programmers “ are tasked with designing and creating software programs, integrating systems and software, training end-users, analyzing algorithms, modifying source-code, writing system instructions, debugging, and maintaining operating systems.” Fortunately, there are solutions to address these issues and one of them is Sentry. Sentry is an error monitoring tool designed to identify and diagnose bugs within your software. Moreover, Sentry is not your only option. There are numerous  alternatives to Sentry  available in the market, providing software engineers with a diverse set of tools to monitor and promptly address crashes in real-time.

What Is Problem-Solving in Programming?

Have you ever encountered a “computer bug?” Maybe you were playing a video game, using an app, or completing work or school assignments on your computer. The bug or glitch became evident when the computer crashed, the screen froze, or the program did not behave as you expected.  

Computer programmers use problem-solving skills, along with an in-depth knowledge of coding, to find and fix issues like these. Each time, the programmer will practice some or all of the following steps .

• Identify the problem. Before a computer program, app, or game is released to the public, it is scrutinized by its makers. Programming teams use the software or play the game, searching for errors. They may recruit others through beta testing programs to use the application and report any problems.
• Understand the problem. Once an error is reported, the programmer must evaluate the issue to determine possible causes and solutions. It’s important to take your time with this step – make sure you really understand every aspect of the problem and your not making assumptions based on past experience.
• Work through the problem. To solve the problem, you’ll need to see it for yourself. Use the application and witness the issue. Then, work through it several more times with different variables to see if the issue is consistent.
• Pseudocode. Write out what you need the lines of code to do before translating it to actual code.
• Translate. Turn the pseudocode into code.
• Test your code. Run the program and find out if it functions as it is supposed to.
• Debug. Fix any errors as you go along.
• Simplify. Errors can arise when code is more complicated than it has to be.
• Take notes. Likely, you won’t be the only one to ever work with your lines of code. Even if you are, you may find that in a few months, you don’t remember exactly what each and every line accomplished. So, take notes on what each line is for.
• Ask for feedback. Just as testing may have revealed the problem in the first place, so it can aid you in identifying any additional issues. Other coders, developers, or programmers may see solutions you don’t. They can make suggestions that will improve the product or process overall.

Improve Your Problem-Solving Skills

Many universities offer online courses that can help you learn creative problem-solving skills that relate directly to information technology applications.

You can also practice problem-solving techniques in everyday situations. When confronted with a challenge, try the following: 

• Identify the problem.
• Get the facts – research similar problems and ask questions. Practice active listening.
• Find possible solutions – brainstorm. Make backup plans in case solutions don’t work as planned.
• Decide on a plan. Weigh the pros and cons of each possible solution, and choose the one that is best overall. If multiple people are involved in the decision-making process, try to arrive at a decision that meets everyone’s needs.
• Act! Once you’ve decided, implement your plan.
• Look for results. Make observations and talk to others involved. Is your plan achieving the desired results? If not, find more facts and follow up with other possible solutions.

Even problem-solving skills from your daily life will serve you well when challenges arise at work.

How to List Problem-Solving Skills on Your Resume

You can include problem-solving skills on your resume in several ways:

• In your skills list
• In your skills summary or core competencies
• Within your job descriptions

You don’t just have to use the term “problem-solving,” either. Show rather than tell. Could you include statements similar to the following that match your unique qualifications?

• “Assisted a team of 6 in debugging a flagship program two days before the product launch.”
• “Identified 32 discrete lines of code that were in error; debugged, simplified, and optimized the code in question.”
• “Communicated with 24 beta testers from around the world over the course of 5 weeks to ready the program for launch.”

You may also use other terminology related to problem-solving, including creativity, analysis, designing, critical thinking, drawing conclusions, focus, experimenting, evaluation, communication, decision-making, and assessment.

Blog / Time To Code / 6 Ways to Improve Your Programming Problem Solving

6 Ways to Improve Your Programming Problem Solving

Sign up for 7pace newsletter.

I would like to sign up to receive email updates from 7pace. Protected by 7pace's privacy policy .

Reporting is here

Software development is, at its core, all about problem solving.

Think about it.

First, developers need to find a problem they can solve with software. Then, they have to figure out how humans solve that problem. And then, they have to find a way to effectively translate both the problem and the solution into code that a computer can use to solve the problem as well as (or better than) a person.

And then there are all the problems along the way: Working with teams, finding and fixing bugs, meeting delivery deadlines.

Engineers use problem solving skills constantly .

Because of that, if you want to become a better developer, one place to start might be becoming a better problem solver. But that’s easier said than done, and requires a deep understanding of what problem solving is, why it matters, and what it actually takes to improve those skills.

Ready to dive in? Let’s get started.

What Is Problem Solving, and Why Does It Matter?

Have you ever heard this famous Steve Jobs quote?

“Everyone in this country should learn to program a computer because it teaches you to think.”

Jobs was right. Software development is as much about “soft skills” like critical thinking, communication, and problem solving as it is about “hard skills” like writing code.

And so, in the context of software development, problem solving can mean a few different things:

• Creating an application that meets the end user’s goals.
• Communicating effectively with team members to delegate work.
• Finding and fixing bugs in the code.
• Meeting a tight deadline for a client.

There’s only one thing that’s true no matter what problem solving looks like on a given day: It’s an integral part of every step of the software development process.

Why Should Engineers Work On Problem Solving Skills?

Just like any other skill, problem solving takes practice to apply and master.

Many developers think that becoming a better problem solver means being able to solve more problems, faster. But that’s not true — it means being able to find the best solution to a problem, and then put that solution in place.

Learning to do that is a great way to become a better developer overall. And while soft skills can be more difficult to learn and improve upon than hard skills, there are still some tips and tricks that can help you get better at problem solving specifically.

6 Ways to Get Better at Problem Solving

As you’ll see from these learning tools, getting better at problem solving is mostly like getting better at any other skill for work: You need to practice. A lot. And then practice some more.

Solve a Lot of Problems on a Lot of Different Platforms

Step one? Solve as many problems as you can, but try to focus on different types of problems on different platforms.

Here’s why this is so beneficial: It prevents you from getting comfortable with one problem solving method or framework. As we already know, in the world of software development, there is definitely no one-size-fits-all solution for the problems we encounter.

When you regularly practice solving different types of problems in different platforms, it reinforces the fact that you can’t always rely on the same technique to solve every problem. It forces you to learn to be flexible, and to choose the best tool or framework for each job.

Solve Problems in Contexts Other Than Work

Since problem solving is a skill that requires practice, you can (and should) work on it even outside of work hours.

This doesn’t need to be a chore — there are a lot of fun ways to practice problem solving, like by doing math or logic puzzles, solving crosswords, or playing a game like chess. Even many video games can help work on problem solving skills.

There are also many opportunities to practice problem solving just as you live your life from day to day. Broke something around the house? Use your problem solving skills to DIY a fix. Need to solve a conflict with a friend or a family member? You guessed it — time to practice problem solving.

Learn From Past Solutions, and Apply Them to New Problems

As you keep practicing problem solving as much as possible, you’ll start to see patterns emerge in the problems you solve. You’ll build up a sort of toolkit filled with the solutions you’ve found and used in the past, and you’ll be able to apply those to solving new problems.

This part is just as important as finding the solutions in the first place, because the more you practice your growing problem solving skills, the more natural it will become to apply the right solutions to different types of problems, making you able to solve new problems more and more quickly, while still using the best possible solves.

Ask Others for Help and Feedback

Sometimes, finding the best solution to a problem just requires a fresh, new set of eyes. That’s why it’s important to treat growing your problem solving skills not as a totally solo venture, but as a team endeavor where everyone at your organization can support each other and help each other get better.

If you’re stuck on a specific problem, ask for help. Someone else might have a method or framework you aren’t familiar with, that they can teach you. You can then apply that to more problems down the road.

And if you’ve come up with a solve for a problem, ask others for feedback. They might be able to help you refine or further improve your framework, making it even better.

Train the Problem Solving Part of Your Mind

How do you keep muscles from growing weaker over time? You keep exercising them.

The same goes for your brain, and especially for different knowledge-base skills, like problem solving. You’ll stay at the top of your brain if you keep “working out,” or practicing problem solving all the time.

A good move for a developer who wants to invest in their problem solving skills is scheduling time every week (or even every day) to consciously practice problem solving. Remember, this doesn’t necessarily mean solving work problems. You could commit to doing a tricky logic puzzle every day on your lunch break, for example. The important thing is to get in the practice, no matter how that looks.

Practice Other Skills Related to Problem Solving

Problem solving is an important skill on its own. But there are other necessary skills developers need to support their problem solving abilities, and those skills all take practice, too.

Flexibility. Critical thinking. Communication. Teamwork. Focusing on building and practicing all these skills will help you improve your problem solving.

Problem solving is one of the most necessary skills for developers to have. With time, practice, and dedication, they can improve it, constantly, and keep becoming better.

Rethinking Timekeeping for Developers:

Turning a timesuck into time well spent.

Leave a Comment

By submitting this form I confirm that I have read the privacy policy and agree to the processing of my personal data for the above mentioned purposes.

Great article regarding problem solving skill, informative and motivating both.

Codility Tests

Outstanding post, I believe people should larn a lot from this website, its really user pleasant.

Technical Screening Tools

I was very happy to discover this great site. I need to thank you for your time just for this fantastic read!

Sharifa Ismail Yusuf

I learnt from this article that one of the key skills a developer need to have is the \"problem solving skills\". Developers also need dedication, time, create time to practice so they can improve their problem solving skills constantly. I do ask for help from others and learn from past solutions and apply them to new problems. From what I have learnt so far, I will try my best to start focusing on building and practicing Flexibility, critical thinking, communication and team work. Solve a lot of problems on a lot of different platforms. Solve problems on context other than work. To start carring out the above, I will schedule time in a week or everyday to conciously practice problem solving skills and other related problem solving skills.Thanks alot for this wonderful article!

dewayne sewell

Ive learnt the skill of problem solving is like a muscle, where it is important to keep exercising it to stay strong. It is important to be aware of the soft skills necessary for effective problem solving also, such as communication, critical thinking, team working that can leverage your technical hard skills to find a solution faster/more effective. Two things I will aim to do is; 1. To solve problems on different platforms so I don’t get too comfortable on only one and stagnate. This not only challenges the brain to see things from a new perspective, but to start the habit of continuous learning and skill building. 2. Reach out to others for help / discuss problems and my solutions for feedback and advice and sharing ideas.

Pakize Bozkurt

Problem solving skills is a crucial thing to make easier or better your life. In fact as a human being we do it in every day life. I mean, we have to do it for living. There are many ways to how to do it. The best way is we should ask right questions. First of all, we should ask some questions, such as; \' Are we aware of the problem?, Are we clarify the problem? Do we go into problem rational? Do we have reasons? or Do we have evidences? Do we do check them out? etc. I am from Philosophy teacher background. I like solving problem whatever in my work or daily life. Secondly, we should have more perspectives . Although our brain is lazy, it is always in a starvation for knowledge.For this there are many enjoyable things to do it. I highly recommend to read book every day which kind of you like it and playing game or solving puzzle. I love solving Sudoku, puzzle and reading book or article. Finally, solving problem is our invatiable needed. Having flexibility, critical thinking, communication and teamwork are easy way to improve us to how we can do our work better and good life. Massive thank for this amazing article!

I read this amazing article. Normally, everyone knows that but we dont use most of time this informations. Which one is the best way to use? Really it does not matter, every one is like a gold opinion. We can use this ideas for the daily life. I have already used that learn from past solution and ask to someone who knows very well. This is so helpful for me. Sometimes google is the best option for ask to someone. Google can be the best teacher for us as well. Soft skills like a team work or solving problem and critical thinking can be important than typing code. We can learn typing code but we can not learn critical thinking and solving problems from google very well. Thank you for this article.

Ipsa iure sed rerum

Excepturi quo volupt

Thanks for this !

Fahil kiima

Thanks a lot for the ideas on problem solving,I really had a problem with that and now going to use what you\'ve informed us about to better my problem solving skills. Thanks.

Alan Codinho

Nice overview

7pace is coming to GitHub! Sign up here for early access to test our beta!

Time tracking can actually be valuable for your team and your organization. But first, you and all your team members need a complete shift in the way you frame time tracking as part of your work.

Sign up for our newsletter and get your free ebook!

Your information is protected by 7pace's privacy policy .

Thanks for subscribing!

Click the download button to receive your free copy of Rethinking Timekeeping for Developers:Turning a Timesuck Into Time Well Spent

Click the download button to receive your free copy of

Contact sales

Please, note that your personal data provided via the above form will be processed in line with the  Privacy Policy . By clicking “Send”, you confirm that you have read the  Privacy Policy  that sets out the purposes for which we process personal data, as well as your rights related to our processing of your personal data.

I wish to receive marketing emails from Appfire.

Request sent

Your message has been transmitted to 7pace.

We will contact you as soon as possible.

Sign up for GitHub News

The 4 best programming languages to learn

Learning to code isn’t just about mastering syntax; it’s a gateway to a lucrative career in software development. According to the U.S. Bureau of Labor Statistics , software developers earned a median annual wage exceeding $127,000 in 2022 and is projected to rise by 25% by 2032.

Rod Garcia, VP of engineering at Slack , succinctly captures the allure: “It’s awesome—the way humans can communicate with machines.” This communication between humans and computers happens through programming languages. Even though the two terms have subtle differences —they are often used interchangeably.

Syracuse University's Online M.S. in Computer Science

Choosing where to begin is like selecting a real-life language to learn. According to the government-run website ShareAmerica, there are between 350 and 430 languages spoken in the United States alone. Similarly, there are hundreds of programming languages out there to choose from. 

Whether it’s Python’s versatility, JavaScript’s ubiquity, or the elegance of SQL, your choice will shape your journey into the intricate world of code. Some languages, like the meme-filled LOLCODE , live in relative obscurity, while others are in high demand, like the leading players JavaScript and Python . 

4 programming languages that are worth learning 

Gone are the days when the best language for beginners was solely determined by the number of available books. Thanks to AI, any language can be a starting point if you possess the determination and requisite skill set to thrive. Ultimately, there’s no one-size-fits-all answer when trying to choose which programming language to learn first. Instead, consider your goals, interests, and the specific problem you aim to solve.

Another important factor to consider is your drive to learn and follow through. “That’s the fuel,” Garcia says. With access to so many resources online, ChatGPT, and AI chatbots, learning a language is becoming easier and easier. “It’s about the fuel you have to want to learn.”

Our methodology in determining the best programming language for beginners draws from two critical sources: the insights provided by Garcia and Stack Overflow’s 2023 Developer Survey , which garnered responses from more than 90,000 professionals.

1. JavaScript

JavaScript simply makes web pages interactive and is also where many other languages stem from. There are more than 80 JavaScript frameworks, including Node.js, Dart, and TypeScript. 

“Not only is [JavaScript] the front-end language of the web, it allows folks to create back-end solutions. If you want to learn one language, JavaScript is great in the early stages of learning how to create full-stack solutions,” Garcia says. 

Classification : JavaScript is a high-level object-oriented scripting language.

Usage : JavaScript allows you to manipulate HTML and CSS, allowing you to create smooth animations, transitions, and visual effects. On the technical side, JavaScript can validate email addresses in web forms. It can also be used to create web-based games, such as Words With Friends 2 and 2048. And with newer frameworks like React Native and Ionic, you can use JavaScript outside the web to create apps for iOS and Android. 

Places to learn it : Fortune found intense bootcamps to teach you how to code JavaScript for less than $21,000. You can learn for free on sites like Udemy , Coursera, and Codecademy. 

Python is the most popular programming language on  GitHub  because of its robust ecosystem of libraries and its general enough nature to fill a variety of programming needs across AI, such as machine learning, deep learning, and computer vision. Its libraries are collections of open-source code modules, which save developers time and effort.

Garcia says that Python has been one of the top languages in the industry for a long time, “but now more than ever, it’s paramount to interact with AI applications or train your own models. It’s the default choice.” According to the World Economic Forum’s 2020 “The Future of Jobs Report,” the artificial intelligence boom will create around 97 million new jobs. 

Classification : Python is a high-level object-oriented scripting language

Usage : Back-end web developers use Python to create web applications, analyze data, and automate tasks. Companies in various industries use it—NASA, Google, and Netflix—to handle big data, perform complex mathematical calculations, and read and modify large files. 

Places to learn it : Bootcamps like General Assembly and CodingNomads teach the language, and you can also learn via the official Python website . Codeacademy’s free course, Python 2 , and The Hitchhiker’s Guide to Python are free resources to learn. Beginner projects include creating secure password generators and games like Rock, Paper, Scissors.

Structured query languages or SQL (pronounced “sequel”) give both analysts and programmers access to data stored within databases. Programmers call SQL “Turing complete,” which indicates a language's power. The more powerful a language, the more complex equations it can perform. 

SQL gives developers access to common table expressions (CTEs) and Window functions (such as SUM, AVG, and COUNT), making it a powerful database management system. It’s widely used by mid-to large-sized organizations, such as the tech giants Facebook and Microsoft, to organize and retrieve information. 

Classification : SQL is a high-level data-oriented language. 

Usage : For developers creating mobile or web apps, when a new user inputs their username and password, it goes into a database—most commonly MySQL, PostgreSQL, and SQLite—and then retrieved with an SQL query.  

However, many different industries use SQL. For example, marketing teams can create promotions targeting customers based on data stored in databases and retrieved with an SQL query. Financial organizations can organize sales data with the language, and healthcare clinics can build dashboards to organize patient reports.

Places to learn it : Codecademy and DataCamp provide courses for learners of all levels. Additionally, eCornell , the online platform affiliated with Cornell University, offers certification programs in data management using SQL and data visualization using Tableau.

Java, an almost three-decade-old programming language, remains one of the most popular choices for developers. It is widely used in customer relationship management (CRM) software applications and financial management platforms because of its robustness, cross-platform capabilities, and security. According to IBM , the language is easy to learn, write, compile, and debug. 

Garcia says Java is used in large enterprise organizations: “Where there are complex legacy systems, there is a certain special type of language you need to manage those systems. Java is one of the most used architecture languages in that context.”

With Java’s Abstract Class feature, you can hide intricate, outdated code to only see the necessary information you need. This tool helps you maintain and manage older systems. According to Garcia, this hideaway capability is a massive part of dealing with old code. “It helps engineers abstract complexities.” 

Classification : Java is a high-level, object-oriented language. 

Usage : Java is a backend development tool used to build web applications with common ones, including Spotify and X (formerly Twitter). Tools like Spring Framework and Hibernate ORM make it easier to construct these newer applications using Java. Sectors like AI, fintech, e-commerce, and gaming benefit from the robustness of language. 2023’s most popular video game, Minecraft, sold more than 300 million copies and was built with Java. 

Places to learn it : Sites like Codecademy and Coursera offer ways to learn Java—each teaching topics like design patterns, data structures, and simple algorithms like sorting. Devmountain’s part-time, remote Java Software Engineering Course costs $9,900. The 16-week program covers Java, Git, frameworks, data structures, and SQL. 

Difference between front-end and back-end programming languages 

There are different types of programming languages, but understanding the difference between front-end and back-end languages is essential for anyone interested in web development.

• Front-end languages involve user interfaces and client-side development. ThinkHTML, CSS, and JavaScript.Front-end languages create the site you’re looking at right now. 
• Back-end languages handle server-side logic and data processing. Think Java, Python, and Ruby on Rails.

Difference between high-level and low-level programming 

The other two main categories for programming languages are high-level and low-level. 

• Despite the name, high-level languages are designed for human readability and ease of understanding. They can run on any platform and are widely used.
• Low-level languages are closer to machine code, less human-friendly, and more efficient for specific tasks. However, they are machine-dependent and less commonly used.

So, what is the best programming language to learn?  

There isn’t a definitive answer to the question, but Garcia argues that you have to start somewhere. 

“Programming languages share common structures, have similar workflows, patterns, and things like that,” Garcia says. “Once you get more experience, you’ll start seeing common themes across languages. Motivation will propel you to learn and go deep.”

There are thousands of languages to learn and lines of code to type—so it may sound daunting to take the first step. But learning something new can be exciting, so focus on the problem you want to solve and then find the language that best suits you, he advises.

For more information on software development skills to get you hired, Fortune compiled the most in-demand programming languages for 2024 .

MBA rankings

• Best Online MBA Programs for 2024
• Best Online Master’s in Accounting Programs for 2024
• Best MBA Programs for 2024
• Best Executive MBA Programs for 2024
• Best Part-Time MBA Programs for 2024
• 25 Most Affordable Online MBAs for 2024
• Best Online Master’s in Business Analytics Programs for 2024

Information technology & data rankings

• Best Online Master’s in Data Science Programs for 2024
• Most Affordable Master’s in Data Science for 2024
• Best Master’s in Cybersecurity Degrees for 2024
• Best Online Master’s in Cybersecurity Degrees for 2024
• Best Online Master’s in Computer Science Degrees for 2024
• Best Master’s in Data Science Programs for 2024
• Most Affordable Online Master’s in Data Science Programs for 2024
• Most Affordable Online Master’s in Cybersecurity Degrees for 2024

Health rankings

• Best Online MSN Nurse Practitioner Programs for 2024
• Accredited Online Master’s of Social Work (MSW) Programs for 2024
• Best Online Master’s in Nursing (MSN) Programs for 2024
• Best Online Master’s in Public Health (MPH) Programs for 2024
• Most Affordable Online MSN Nurse Practitioner Programs for 2024
• Best Online Master’s in Psychology Programs for 2024

Leadership rankings

• Best Online Doctorate in Education (EdD) Programs for 2024
• Most Affordable Online Doctorate in Education (EdD) Programs for 2024
• Coding Bootcamps in New York for 2024
• Best Data Science and Analytics Bootcamps for 2024
• Best Cybersecurity Bootcamps for 2024
• Best UX/UI bootcamps for 2024

Boarding schools

• World’s Leading Boarding Schools for 2024
• Top Boarding School Advisors for 2024

Berkeley's Data Science Master's

Welcome to Java! Easy Max Score: 3 Success Rate: 97.06%

Java stdin and stdout i easy java (basic) max score: 5 success rate: 96.89%, java if-else easy java (basic) max score: 10 success rate: 91.32%, java stdin and stdout ii easy java (basic) max score: 10 success rate: 92.61%, java output formatting easy java (basic) max score: 10 success rate: 96.55%, java loops i easy java (basic) max score: 10 success rate: 97.68%, java loops ii easy java (basic) max score: 10 success rate: 97.32%, java datatypes easy java (basic) max score: 10 success rate: 93.66%, java end-of-file easy java (basic) max score: 10 success rate: 97.91%, java static initializer block easy java (basic) max score: 10 success rate: 96.16%.

• Trending Now
• Foundational Courses
• Data Science
• Practice Problem
• Machine Learning
• System Design
• DevOps Tutorial
• Web Browser

Welcome to the daily solving of our PROBLEM OF THE DAY with Nitin Kaplas . We will discuss the entire problem step-by-step and work towards developing an optimized solution. This will not only help you brush up on your concepts of Tree but also build up problem-solving skills. In this problem, we are given Red lines of slope -1 passing between nodes (in the following diagram). The diagonal sum in a binary tree is the sum of all node datas lying between these lines. Given a Binary Tree of size n, print all diagonal sums. For the following input tree, output should be 9, 19, 42. 9 is sum of 1, 3 and 5. 19 is sum of 2, 6, 4 and 7. 42 is sum of 9, 10, 11 and 12.

DiagonalSum Example : Input:         4       /   \      1     3           /          3 Output:  7 4  Give the problem a try before going through the video. All the best!!! Problem Link: https://www.geeksforgeeks.org/problems/diagonal-sum-in-binary-tree/1

• Skip to primary navigation
• Skip to main content
• Skip to primary sidebar
• Skip to footer

Codemotion Magazine

We code the future. Together

Devin: a New End-to-end AI Programming Tool

A new AI coding tool enters the race: let’s take a closer look at Devin, a new AI coding assistant (currently in early access) that has already been nicknamed the “First AI Software Engineer”.

March 18, 2024 by Codemotion

Developed by Cognition, a pioneering US startup, Devin emerges as the world’s premier fully autonomous AI coding tool. But what sets Devin apart from its predecessors like chatGPT and GitHub Copilot ? Let’s navigate through the realm of AI coding assistants to uncover the distinctive capabilities that define Devin.

The Current AI Coding Assistant Scene

Recent years have witnessed a surge in AI-driven coding assistants. Tools such as chatGPT and GitHub Copilot have become indispensable aids for programmers, offering code generation, completion suggestions, and error detection. However, these tools operate within the defined boundaries of preexistent LLMs , excelling in streamlining repetitive tasks or providing insights based on existing data. These tools, despite their original limitations, are being adopted by companies and developers at an outstanding rate, and are even starting to replace the traditional code-solving references such as Stack Overflow.

Introducing Devin: The Comprehensive AI Engineer

According to their creators, Devin transcends the limitations of its counterparts, offering a holistic approach to AI coding assistance. Unlike other tools, Devin isn’t confined to mere code suggestions; it’s engineered to tackle entire development projects from inception to fruition. It doesn’t just suggest code and find bugs, it basically builds the final output on its own.

Here’s what sets Devin apart:

End-to-End Development : Devin possesses the capability to transform project outlines into fully functional applications, encompassing tasks ranging from requirement comprehension to code composition, bug resolution, and application deployment.

Autonomous Learning : Operating autonomously, Devin harnesses resources like API documentation and online tutorials to assimilate knowledge and complete tasks independently.

Self-Sufficient Problem-Solving: Beyond code generation, Devin adeptly tackles intricate engineering challenges, devising solutions within its secure sandbox environment.

A Comparative Analysis: Devin vs. Traditional AI Coding Assistants

Implications: advancement or adversity for programmers.

Devin’s prowess prompts a pivotal question: does it aim to supplant human programmers? While Devin streamlines numerous tasks, it’s unlikely to eliminate the need for human expertise due to:

Domain Knowledge : Programming demands a profound comprehension of the problem domain, an aspect where Devin may lack the nuanced understanding possessed by seasoned human programmers.

Creativity and Innovation : Programmatic endeavors often necessitate inventive problem-solving, an area where Devin, despite its learning capabilities, may encounter challenges requiring unconventional approaches.

Human Oversight : Critical aspects such as testing, debugging, and ensuring code quality remain reliant on human intervention.

Collaboration or Bane?

The rise of devin: a boon or bane for programmers.

Devin’s arrival throws gasoline on the already heated debate surrounding AI and its potential impact on the programming profession. Unlike its predecessors, Devin isn’t confined to code completion or bug identification. It aspires to create entire products, autonomously searching for and applying fixes, a capability that sets it apart from existing tools. This newfound level of automation has some fearing a future where AI replaces human programmers altogether.

The Potential Benefits: A Programmer’s Dream Team Player?

Proponents of Devin paint a rosy picture. By automating repetitive tasks like bug fixing and code generation, Devin can become a programmer’s dream team player. Here’s how:

• Enhanced Developer Productivity: Imagine a world where programmers are freed from the drudgery of repetitive tasks. Devin can handle those, allowing programmers to focus on the strategic aspects of software development – design, architecture, and the creative problem-solving that truly sets them apart.
• Accelerated Development: Devin’s ability to work autonomously has the potential to significantly reduce development timelines. This translates to faster product launches and quicker time-to-market advantages for businesses.
• Democratization of Programming: Devin’s user-friendly interface could potentially open doors for non-programmers to dabble in application development. This could foster greater innovation and lead to the creation of entirely new categories of software.

The Concerns: A Job Market in Flux?

However, Devin’s rise also raises valid concerns:

• Job displacement: Some fear that AI tools like Devin could automate programmers out of a job entirely. While repetitive tasks might be at risk, the need for human expertise in areas like design, problem-solving, and domain knowledge is unlikely to disappear.
• Ethical considerations: Bias in AI code is a well-documented concern. If Devin is left unchecked, it could perpetuate or even amplify existing biases in software.
• Impact on the workforce: The transition to a more AI-driven development landscape will require workforce retraining and adaptation. How the programming job market adapts to this new reality remains to be seen.

While Devin and other AI tools don’t signify the obsolescence of human programmers, they usher in an era of symbiotic collaboration and continuous learning. Companies should continue to train their IT employees to understand and leverage new AI tools responsibly, acknowledging their limitations, risks, and creating best practices.

AI coding tools like Devin are here to empower programmers, not replace them. Imagine a future where programmers spend less time on repetitive tasks and more time on innovation, tackling complex challenges, and pushing the boundaries of software development. With Devin (or any other AI coding tool you like) as a partner, programmers can focus on the creative spark, the human touch that truly differentiates exceptional software.

Angular Signal Queries: simplifying DOM querying

Don’t let your github project die: how to improve engagement and attract contributors, related articles.

• Top Affordable AI Certifications For Boosting Your Career
• Generative Video Creation: Sora by OpenAI is Here
• How Can Synthetic Data Help in Search and Rescue Missions?
• Become a partner
• Tech articles
• Discover talent
• Discover companies

For Business

• Codemotion for companies
• Become a contributor
• Work with us

Accelerated first-order methods for a class of semidefinite programs

• Full Length Paper
• Published: 22 March 2024

Cite this article

• Alex L. Wang   ORCID: orcid.org/0000-0002-4293-0359 1 , 2 &
• Fatma Kılınç-Karzan 1  

This paper introduces a new storage-optimal first-order method, CertSDP, for solving a special class of semidefinite programs (SDPs) to high accuracy. The class of SDPs that we consider, the exact QMP-like SDPs , is characterized by low-rank solutions, a priori knowledge of the restriction of the SDP solution to a small subspace, and standard regularity assumptions such as strict complementarity. Crucially, we show how to use a certificate of strict complementarity to construct a low-dimensional strongly convex minimax problem whose optimizer coincides with a factorization of the SDP optimizer. From an algorithmic standpoint, we show how to construct the necessary certificate and how to solve the minimax problem efficiently. Our algorithms for strongly convex minimax problems with inexact prox maps may be of independent interest. We accompany our theoretical results with preliminary numerical experiments suggesting that CertSDP significantly outperforms current state-of-the-art methods on large sparse exact QMP-like SDPs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA) Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Technically, these papers establish that the optimal values or optimal solutions of the SDP relaxation coincide with that of the underlying QCQP. Nonetheless, many of these sufficient conditions prove the intermediate result of strict complementarity.

In [ 76 ], a rank- k matrix \({\tilde{Y}}\in {\mathbb {S}}^n_+\) is a \((1+\zeta )\) -optimal rank- k approximation of an \(\epsilon \) -optimal solution \(Y_\epsilon \in {\mathbb {S}}^n_+\) if \(\left\Vert Y_\epsilon - {\tilde{Y}} \right\Vert _* \le (1+\zeta ) \left\Vert Y_\epsilon - [Y_\epsilon ]_k \right\Vert _*\) where \(\left\Vert \cdot \right\Vert _*\) is the nuclear norm and \([Y_\epsilon ]_k\) is the best rank- k approximation of \(Y_\epsilon \) .

It is in fact true that the two sets are equal but only one direction is necessary in this proof.

Abbe, E., Bandeira, A.S., Hall, G.: Exact recovery in the stochastic block model. IEEE Trans. Inform. Theory 62 (1), 471–487 (2015)

Article   MathSciNet   Google Scholar  

Alizadeh, F.: Interior point methods in semidefinite programming with applications to combinatorial optimization. SIAM J. Optim. 5 (1), 13–51 (1995)

Alizadeh, F., Haeberly, J.A., Overton, M.L.: Complementarity and nondegeneracy in semidefinite programming. Math. Program. 77 , 111–128 (1997)

Argue, C.J., Kılınç-Karzan, F., Wang, A.L.: Necessary and sufficient conditions for rank-one generated cones. Math. Oper. Res. 48 (1), 100–126 (2023)

Baes, M., Burgisser, M., Nemirovski, A.: A randomized mirror-prox method for solving structured large-scale matrix saddle-point problems. SIAM J. Optim. 23 (2), 934–962 (2013)

Beck, A.: Quadratic matrix programming. SIAM J. Optim. 17 (4), 1224–1238 (2007)

Beck, A., Drori, Y., Teboulle, M.: A new semidefinite programming relaxation scheme for a class of quadratic matrix problems. Oper. Res. Lett. 40 (4), 298–302 (2012)

Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization, volume 2 of MPS-SIAM Series on Optimization. SIAM (2001)

Ben-Tal, A., Nemirovski, A.: Solving large scale polynomial convex problems on \(\ell _1\) /nuclear norm balls by randomized first-order algorithms. In: CoRR (2012)

Boumal, N., Voroninski, V., Bandeira, A.: The non-convex Burer–Monteiro approach works on smooth semidefinite programs. In: Advances in Neural Information Processing Systems, vol. 29 (2016)

Burer, S., Kılınç-Karzan, F.: How to convexify the intersection of a second order cone and a nonconvex quadratic. Math. Program. 162 , 393–429 (2017)

Burer, S., Monteiro, R.D.C.: A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization. Math. Program. 95 , 329–357 (2003)

Burer, S., Yang, B.: The trust region subproblem with non-intersecting linear constraints. Math. Program. 149 , 253–264 (2014)

Burer, S., Ye, Y.: Exact semidefinite formulations for a class of (random and non-random) nonconvex quadratic programs. Math. Program. 181 , 1–17 (2019)

Candès, E.J., Eldar, Y.C., Strohmer, T., Voroninski, V.: Phase retrieval via matrix completion. SIAM Rev. 57 (2), 225–251 (2015)

Carmon, Y., Duchi, J.C.: Analysis of Krylov subspace solutions of regularized nonconvex quadratic problems. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp. 10728–10738 (2018)

Chambolle, A., Pock, T.: On the ergodic convergence rates of a first-order primal-dual algorithm. Math. Program. 159 , 253–287 (2016)

Cifuentes, D.: On the Burer–Monteiro method for general semidefinite programs. Opt. Lett. 15 (6), 2299–2309 (2021)

Cifuentes, D., Moitra, A.: Polynomial time guarantees for the Burer–Monteiro method. Adv. Neural. Inf. Process. Syst. 35 , 23923–23935 (2022)

Google Scholar  

d’Aspremont, A., El Karoui, N.: A stochastic smoothing algorithm for semidefinite programming. SIAM J. Optim. 24 (3), 1138–1177 (2014)

de Carli-Silva, M.K., Tunçel, L.: Strict complementarity in semidefinite optimization with elliptopes including the maxcut SDP. SIAM J. Optim. 29 (4), 2650–2676 (2019)

Devolder, O., Glineur, F., Nesterov, Y.: First-Order Methods with Inexact Oracle: The Strongly Convex Case. Technical Report 2013016 (2013)

Devolder, O., Glineur, F., Nesterov, Y.: First-order methods of smooth convex optimization with inexact oracle. Math. Program. 146 (1), 37–75 (2014)

Ding, L., Udell, M.: On the simplicity and conditioning of low rank semidefinite programs. SIAM J. Optim. 31 (4), 2614–2637 (2021)

Ding, L., Wang, A.L.: Sharpness and well-conditioning of nonsmooth convex formulations in statistical signal recovery (2023). arXiv:2307.06873

Ding, L., Yurtsever, A., Cevher, V., Tropp, J.A., Udell, M.: An optimal-storage approach to semidefinite programming using approximate complementarity. SIAM J. Optim. 31 (4), 2695–2725 (2021)

Drusvyatskiy, D., Lewis, A.S.: Error bounds, quadratic growth, and linear convergence of proximal methods. Math. Oper. Res. 43 (3), 919–948 (2018)

Fradkov, A.L., Yakubovich, V.A.: The S-procedure and duality relations in nonconvex problems of quadratic programming. Vestnik Leningrad Univ. Math. 6 , 101–109 (1979)

Friedlander, M.P., Macêdo, I.: Low-rank spectral optimization via gauge duality. SIAM J. Sci. Comput. 38 (3), A1616–A1638 (2016)

Garber, D., Kaplan, A. A.: On the efficient implementation of the matrix exponentiated gradient algorithm for low-rank matrix optimization. Math. Oper. Res. (2022)

Goemans, M.X., Williamson, D.P.: Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. J. ACM 42 (6), 1115–1145 (1995)

Goldfarb, D., Scheinberg, K.: Interior point trajectories in semidefinite programming. SIAM J. Optim. 8 (4), 871–886 (1998)

Hamedani, E.Y., Aybat, N.C.: A primal–dual algorithm with line search for general convex–concave saddle point problems. SIAM J. Optim. 31 (2), 1299–1329 (2021)

Hazan, E., Koren, T.: A linear-time algorithm for trust region problems. Math. Program. 158 , 363–381 (2016)

Ho-Nguyen, N., Kılınç-Karzan, F.: A second-order cone based approach for solving the Trust Region Subproblem and its variants. SIAM J. Optim. 27 (3), 1485–1512 (2017)

Jeyakumar, V., Li, G.Y.: Trust-region problems with linear inequality constraints: exact SDP relaxation, global optimality and robust optimization. Math. Program. 147 , 171–206 (2014)

Jiang, R., Li, D.: Novel reformulations and efficient algorithms for the Generalized Trust Region Subproblem. SIAM J. Optim. 29 (2), 1603–1633 (2019)

Juditsky, A., Nemirovski, A.: First order methods for non-smooth convex large-scale optimization, ii: utilizing problems structure. Optim. Mach. Learn. 30 (9), 149–183 (2011)

Kılınç-Karzan, F., Wang, A.L.: Exactness in SDP relaxations of QCQPs: theory and applications. Tut. Oper. Res. Informs (2021)

Lan, G., Lu, Z., Monteiro, R.D.C.: Primal–dual first-order methods with \(o(1/\epsilon )\) iteration-complexity for cone programming. Math. Program. 126 , 1–29 (2011)

Laurent, M., Poljak, S.: On a positive semidefinite relaxation of the cut polytope. Linear Algebra Appl. 223–224 , 439–461 (1995)

Levy, K.Y., Yurtsever, A., Cevher, V.: Online adaptive methods, universality and acceleration. In: Advances in Neural Information Processing Systems (2018)

Locatelli, M.: Exactness conditions for an SDP relaxation of the extended trust region problem. Oper. Res. Lett. 10 (6), 1141–1151 (2016)

MathSciNet   Google Scholar  

Locatelli, M.: KKT-based primal–dual exactness conditions for the Shor relaxation. J. Glob. Optim. 86 (2), 285–301 (2023)

Lu, Z., Nemirovski, A., Monteiro, R.D.C.: Large-scale semidefinite programming via a saddle point mirror-prox algorithm. Math. Program. 109 , 211–237 (2007)

Majumdar, A., Hall, G., Ahmadi, A.A.: Recent scalability improvements for semidefinite programming with applications in machine learning, control, and robotics. Annu. Rev. Control Robot. Auton. Syst. 3 , 331–360 (2020)

Article   Google Scholar  

Mixon, D.G., Villar, S., Ward, R.: Clustering subgaussian mixtures by semidefinite programming. Inf. Inference J. IMA 6 (4), 389–415 (2017)

Moré, J.J.: Generalizations of the Trust Region Problem. Optim. Methods Softw. 2 (3–4), 189–209 (1993)

Moré, J.J., Sorensen, D.C.: Computing a trust region step. SIAM J. Sci. Stat. Comput. 4 , 553–572 (1983)

Nemirovski, A.: Prox-method with rate of convergence o(1/t) for variational inequalities with Lipschitz continuous monotone operators and smooth convex–concave saddle point problems. SIAM J. Optim. 15 (1), 229–251 (2004)

Nesterov, Y.: Excessive gap technique in nonsmooth convex minimization. SIAM J. Optim. 16 (1), 235–249 (2005)

Nesterov, Y.: Smooth minimization of non-smooth functions. Math. Program. 103 , 127–152 (2005)

Nesterov, Y.: Lectures on convex optimization. Springer Optimization and Its Applications, vol. 137. Springer (2018)

Nesterov, Y., Nemirovskii, A.: Interior-Point Polynomial Algorithms in Convex Programming. SIAM, Philadelphia (1994)

Book   Google Scholar  

O’Donoghue, B., Chu, E., Parikh, N., Boyd, S.: Conic optimization via operator splitting and homogeneous self-dual embedding. J. Optim. Theory Appl. 169 (3), 1042–1068 (2016)

Ouyang, Y., Xu, Y.: Lower complexity bounds of first-order methods for convex–concave bilinear saddle-point problems. Math. Program. 185 , 1–35 (2021)

Palaniappan, B., Bach, F.: Stochastic variance reduction methods for saddle-point problems. In: Advances in Neural Information Processing Systems, vol. 29 (2016)

Raghavendra, P.: Optimal algorithms and inapproximability results for every CSP? In: Proceedings of the Fortieth Annual ACM Symposium on Theory of Computing, pp. 245–254 (2008)

Rujeerapaiboon, N., Schindler, K., Kuhn, D., Wiesemann, W.: Size matters: cardinality-constrained clustering and outlier detection via conic optimization. SIAM J. Optim. 29 (2), 1211–1239 (2019)

Sard, A.: The measure of the critical values of differentiable maps. Bull. Am. Math. Soc. 48 (12), 883–890 (1942)

Shinde, N., Narayanan, V., Saunderson, J.: Memory-efficient structured convex optimization via extreme point sampling. SIAM J. Math. Data Sci. 3 (3), 787–814 (2021)

Shor, N.Z.: Dual quadratic estimates in polynomial and Boolean programming. Ann. Oper. Res. 25 , 163–168 (1990)

Sion, M.: On general minimax theorems. Pac. J. Math. 8 (1), 171–176 (1958)

Souto, M., Garcia, J.D., Veiga, Á.: Exploiting low-rank structure in semidefinite programming by approximate operator splitting. Optimization 1–28 (2020)

Sturm, J.F., Zhang, S.: On cones of nonnegative quadratic functions. Math. Oper. Res. 28 (2), 246–267 (2003)

Tseng, P.: On accelerated proximal gradient methods for convex–concave optimization (2008)

Vandenberghe, L., Boyd, S.: Semidefinite programming. SIAM Rev. 38 (1), 49–95 (1996)

Waldspurger, I., Waters, A.: Rank optimality for the Burer–Monteiro factorization. SIAM J. Optim. 30 (3), 2577–2602 (2020)

Wang, A.L., Kılınç-Karzan, F.: A geometric view of SDP exactness in QCQPs and its applications (2020). arXiv:2011.07155

Wang, A.L., Kılınç-Karzan, F.: The generalized trust region subproblem: solution complexity and convex hull results. Math. Program. 191 (2), 445–486 (2022)

Wang, A.L., Kılınç-Karzan, F.: On the tightness of SDP relaxations of QCQPs. Math. Program. 193 (1), 33–73 (2022)

Wang, A.L., Lu, Y., Kılınç-Karzan, F.: Implicit regularity and linear convergence rates for the generalized trust-region subproblem. SIAM J. Optim. 33 (2), 1250–1278 (2023)

Yang, H., Liang, L., Carlone, L., Toh, K.: An inexact projected gradient method with rounding and lifting by nonlinear programming for solving rank-one semidefinite relaxation of polynomial optimization. Math. Program. 201 (1–2), 409–472 (2023)

Yurtsever, A., Fercoq, O., Cevher, V.: A conditional-gradient-based augmented Lagrangian framework. In: International Conference on Machine Learning, pp. 7272–7281 (2019)

Yurtsever, A., Tropp, J.A., Fercoq, O., Udell, M., Cevher, V.: Scalable semidefinite programming. SIAM J. Math. Data Sci. 3 (1), 171–200 (2021)

Download references

Acknowledgements

This research is supported in part by ONR Grant N00014-19-1-2321 and AFOSR Grant FA9550-22-1-0365. The authors wish to thank the review team for their feedback and suggestions that led to an improved presentation of the material.

Author information

Authors and affiliations.

Carnegie Mellon University, Pittsburgh, PA, USA

Alex L. Wang & Fatma Kılınç-Karzan

Purdue University, West Lafayette, IN, USA

Alex L. Wang

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Alex L. Wang .

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Deferred proofs

Proof of lemma 3.

Let \(\varDelta :={\tilde{X}} - X_L\) . Then,

where the second equality follows from expanding the square and the fact that \(L = {\tilde{L}} + {\tilde{\mu }}\) . Moreover,

Combining these two inequalities gives

\(\square \)

The following proof is adapted from [ 54 ].

Proof of Lemma 4

It is evident that \(\phi _t(X)\) are quadratic matrix functions of the form ( 8 ) with \(V_0=X_0\) and \(\phi _0^*=Q(X_0)\) . The remainder of the proof verifies the recurrences on \(V_{t+1}\) and \(\phi ^*_{t+1}\) . We suppose that the stated form holds for some t , and we will show that it will hold for \(t+1\) as well. We compute

We deduce that \(V_{t+1} = (1-\alpha )V_t + \alpha \left( \varXi _t - \frac{1}{{\tilde{\mu }}}{\tilde{g}}_t\right) \) . Noting that \(\phi _{t+1}^*=\phi _{t+1}(V_{t+1})\) , and applying the recursive definition of \(\phi _{t+1}(X)\) gives us

where the third equation follows from substituting the expression for \(V_{t+1}\) , and the last one from regrouping the terms. \(\square \)

The following proof is adapted from [ 54 , Page 92].

Proof of Lemma 5

Proof of lemma 6.

It is clear that \(Q(X_0)\le \phi _0^*\) . Thus, consider \(X_{t+1}\) with \(t\ge 0\) . By induction and Lemma  4 ,

As \(X_{t+1}\) satisfies \(Q_L(\varXi _t; X_{t+1}) \le Q^*(\varXi _t) + \epsilon _t\) , we deduce (see Theorem  2 ) that

These two inequalities together lead to

It is straightforward to show that the two quantities on the final line are identically zero using the relations \(\alpha ^2 = {\tilde{\mu }} / {\tilde{L}}\) and \(\varXi _t = \tfrac{X_t + \alpha V_t}{1+\alpha }\) (see Lemma  5 ). \(\square \)

Proof of Lemma 7

The statement holds holds for \(t = 0\) . Thus, consider \(\phi _{t+1}\) for \(t\ge 0\) . By definition

Then, these inequalities combined with the inductive hypothesis give

Proof of Corollary 1

Let \(X^*_\mathcal{U}\) denote the optimizer of (QMMP) so that \(Q(X^*_\mathcal{U}) = {{\,\textrm{Opt}\,}}_{\text {(QMMP)}}\) . Then, Lemmas  6 and 7 give

Note also that by the definition of \(\phi _0(\cdot )\) and the \(\mu \) -strong convexity of Q , we have

Combining the two inequalities completes the proof. \(\square \)

Proof of Lemma 8

Let \({\tilde{\gamma }}\in \mathop {\mathrm {arg\,max}}\limits _{\gamma \in \mathcal{U}}q(\gamma ,X_0)\) . By \(\mu \) -strong convexity of Q ( X ), we have that

In particular, taking \(X = \mathop {\mathrm {arg\,min}}\limits _{X\in {\mathbb {R}}^{(n-k)\times k}} Q(X)\) gives

where the last inequality follows from Assumption  4 . This proves the first claim. Next, by Theorem  3 , we have that for all \(t\ge 0\) , that \(Q(X_t)-Q(X_0)\le Q(X_t)-{{\,\textrm{Opt}\,}}_{\text {(QMMP)}} \le 2\mu \kappa ^2 R^2\) and hence

Using the assumption \(X_0 = 0_{(n-k)\times k}\) in Assumption  4 and applying triangle inequality together with the bound \(\left\Vert \nabla _2\,q({\tilde{\gamma }}, X_0) \right\Vert _F^2\le \mu ^2\kappa ^2 R^2\) derived from Assumption  4 , we deduce that for all \(t\ge 0\) ,

Then, as \(\varXi _{t+1} = X_{t+1} + \frac{1-\alpha }{1+\alpha }\left( X_{t+1} - X_t\right) \) , we have

Proof of Lemma 9

Recall that by definition, the linear operator \(\mathcal{G}\) maps \(\gamma \) to \(\sum _{i=1}^m \gamma _i \left( A_i \varXi _t + B_i\right) \) . Thus, for any \(\gamma \in {\textbf{S}}^{m-1}\) ,

Proof of Lemma 10

Let \(r :=\left\Vert \gamma ^{(i)} - \gamma ^* \right\Vert _2\) . Using Assumption  5 , we may bound the individual terms within the definition of \(r^{(i)}\) as

Thus, \(r^{(i)} \ge \min \left( \frac{{\hat{\mu }}}{2{\hat{\rho }}},\,\frac{{\hat{\mu }}}{2{\hat{\rho }}} - r\right) = \frac{{\hat{\mu }}}{2{\hat{\rho }}} - r\) . Then, when \(r\le \frac{{\hat{\mu }}}{4{\hat{\rho }}}\) , we have \(r^{(i)}>0\) and furthermore, \(r^{(i)}\ge r = \left\Vert \gamma ^{(i)} - \gamma ^* \right\Vert _2\) . \(\square \)

Proof of Lemma 11

Begin by noting that for all \(\gamma \in \mathcal{U}^{(i)}\) ,

Let \({\tilde{\gamma }}\in \mathop {\mathrm {arg\,max}}\limits _{\gamma \in \mathcal{U}^{(i)}}q(\gamma ,0_{(n-k)\times k})\) . Then,

Next, for \(\gamma \in {\textbf{S}}^{m-1}\)

Consider an instance of ( 14 ) generated by the random procedure in Sect.  5.2 . Then equality holds throughout ( 14 ).

It suffices to show that \(\gamma ^*\) and \(T^*\) are feasible and achieve value \(\left\Vert X^* \right\Vert _F^2\) in the dual SDP (i.e., the third line of ( 14 )).

Note that by Schur Complement Theorem,

Here, \(\sim \) indicates matrix similarity. Thus, \(\gamma ^*\) and \(T^*\) are feasible in the dual SDP.

Strict complementarity in quadratic matrix programs

In this section, we show that a generic quadratic matrix program (QMP) in an \(n\times k\) dimensional matrix variable with at most k constraints satisfies strict complementarity (assuming only existence of primal and dual solutions).

We will need the following lemma stating that a generic bilinear system has only the trivial solutions. This lemma follows from basic dimension-counting arguments in algebraic geometry. However, we will instead prove the lemma directly using only elementary tools.

Let \(n,p\in {\mathbb {N}}\) and consider the space \(({\mathbb {R}}^{n\times p})^{n+p-1}\) . Let the collection \((A_i) = (A_1,\ldots , A_{n+p-1})\) denote an element of this space. Here, each \(A_i\in {\mathbb {R}}^{n\times p}\) . Then, the collections \((A_i)\) for which the bilinear system

has a nontrivial solution (i.e., where \(x\in {\mathbb {R}}^n\) and \(y\in {\mathbb {R}}^p\) are both nonzero) forms a set of measure zero in \(({\mathbb {R}}^{n\times p})^{n+p-1}\) .

Let \(\mathcal{S}\) be the exceptional set, i.e.,

By homogeneity, we may require that \(x\in {\mathbb {R}}^n\) has some coordinate equal to one. Similarly, we will require that \(y\in {\mathbb {R}}^p\) has some coordinate equal to one. Thus, we may decompose \(\mathcal{S}= \bigcup _{\ell =1}^n\bigcup _{r=1}^p \mathcal{S}_{\ell ,r}\) , where

We will show that for each \(\ell \in [n]\) and \(r\in [p]\) that \(\mathcal{S}_{\ell ,r}\) has measure zero. Without loss of generality, let \(\ell = r= 1\) .

Consider the affine space

Let \(\mathcal{F}_{1,1}: \mathcal{M}\rightarrow ({\mathbb {R}}^{n\times p})^{n+p-1}\) send the element \((x,y,B_1,\ldots ,B_{n+p-1})\) to \((A_1,\ldots ,A_{n+p-1})\) where

One may verify that \(\mathcal{F}_{1,1}\) is a smooth map. Furthermore, its domain has dimension \((n-1) + (p-1) + (np - 1)(n+p-1) = np(n+p-1) - 1\) . This is one less than the dimension of the space \(\left( {\mathbb {R}}^{n\times p}\right) ^{n+p-1}\) . It is known that the image of a Euclidean space under a smooth map into a Euclidean space of higher dimension must have Lebesgue measure zero (see Sard’s lemma [ 61 ]). Thus, \(\mathcal{F}_{1,1}(\mathcal{M})\) has Lebesgue measure zero.

It remains to verify Footnote 3 that \(\mathcal{S}_{1,1}\subseteq \mathcal{F}_{1,1}(\mathcal{M})\) . Suppose \((A_i)\in \mathcal{S}_{1,1}\) and let x ,  y with \(x_1 = y_1 = 1\) satisfy \(x^\intercal A_i y = 0\) for all \(i\in [n+p-1]\) . Let

Note that \((B_i)_{1,1} = \frac{1}{\left\Vert x \right\Vert ^2\left\Vert y \right\Vert ^2}x^\intercal A_i y = 0\) for all \(i\in [n+p-1]\) . The remaining sets \(\mathcal{S}_{\ell ,r}\) can be shown to have measure zero using analogous maps \(\mathcal{F}_{\ell ,r}\) . This concludes the proof. \(\square \)

Let \(n,k\in {\mathbb {N}}\) and consider the SDP relaxation of a QMP with k constraints in a variable of size \(n\times k\) and its dual:

There exists a subset \(\mathcal{E}\subseteq ({\mathbb {S}}^n)^{1+k} \times ({\mathbb {R}}^{n\times k})^{1+k}\) of measure zero such that if

and the primal and dual SDPs are both solvable, then strict complementarity holds and the primal and dual SDPs both have unique optimizers.

We will condition on the following bilinear system in the variables \((\gamma _\text {obj},\ldots ,\gamma _k)\in {\mathbb {R}}^{1+k}\) and \(x\in {\mathbb {R}}^n\) having no nontrivial solutions:

This is a homogeneous bilinear system in \(n + 1 + k\) variables with \(n+k\) constraints. Thus, by Lemma  14 , this system has no nontrivial solutions outside an exceptional set \(\mathcal{E}\) of measure zero.

Let \((\gamma ^*,T^*)\) denote a dual optimum solution. We claim that \(A(\gamma ^*)\succ 0\) . For the sake of contradiction, assume that \(x\in \ker (A(\gamma ^*))\) is nonzero. Then, by assumption, \(x^*\) and \((1,\gamma ^*)\) are not a solution to the bilinear system above, i.e., \(B(\gamma ^*)^\intercal x \ne 0\) and there exists a column of \(B(\gamma ^*)\) , say the first column, that has nonzero inner product with x . This contradicts the feasibility of \((\gamma ^*, T^*)\) . Specifically for \(\alpha \in {\mathbb {R}}\) ,

Picking \(\alpha \) large or small enough makes this quantity negative, contradicting that the matrix on the left is positive semidefinite.

We have shown that for every dual optimum solution \((\gamma ^*, T^*)\) , the above slack matrix has rank at least n . Similarly, any primal optimum solution \(Y^*\) must have rank at least k . We deduce that every primal optimum solution \(Y^*\) has rank exactly k and that for every dual optimum solution \((\gamma ^*, T^*)\) , the slack matrix has rank exactly n . Now, these optimizers must correspond to faces of slices of \({\mathbb {S}}^{n+k}_+\) . As the only faces of slices of \({\mathbb {S}}^{n+k}_+\) with constant rank are singleton sets, we deduce that there is a unique primal optimizer and a unique dual optimizer. \(\square \)

Additional experiments on phase-retrieval inspired SDP instances

We perform additional experiments on SDP instances inspired by the phase retrieval problem.

The phase retrieval problem seeks to learn a vector \(x^*\) given only the magnitudes of linear measurements of \(x^*\) , and finds applications in imaging. In the Gaussian model of phase retrieval [ 15 ], we assume \(x^*\in {\mathbb {R}}^n\) is arbitrary and \(G\in {\mathbb {R}}^{m \times n}\) is entrywise Gaussian with an appropriate normalization. We are given

Here, the absolute value is taken entrywise. Equivalently, we are given the entrywise square of \(Gx^*\) , or \(b = {{\,\textrm{diag}\,}}(Gx^*(x^*)^\intercal G^\intercal )\) . In this setting, it is known that the PhaseLift SDP,

has \((x^*)(x^*)^\intercal \) as its unique solution with high probability once the number of observations m is roughly O ( n ). Recent work [ 25 ] shows that strict complementarity holds between this SDP and its dual with high probability in the same regime.

We note that the Gaussian model of phase retrieval requires storing the matrix G as part of the instance. This is a matrix of size \(O(n^2)\) and thus limits the size of our current experiments. Nonetheless, we expect the behavior we observe with these experiments to hold in the real setting of phase retrieval where the matrix G is highly structured and can be stored implicitly. We leave this as important future work.

We compare CertSDP (Algorithm 2), CSSDP [ 26 ], SketchyCGAL [ 76 ], ProxSDP [ 65 ], SCS [ 56 ], and Burer–Monteiro [ 12 ].

Random instance generation. We generate instances as follows. Suppose n is given. We set \(m = 5n\) . We generate \(G\in {\mathbb {R}}^{m\times n}\) where each entry is independent N (0, 1/ m ). We then preprocess G so that its m th observation vector, i.e., the m th row of G , is parallel to \(e_n\) . Next, we sample \(x^*\) uniformly form

Thus, this is a random instance of phase retrieval where we are given one highly-correlated observation.

Implementation details. The algorithms we test are mostly as described in Sect.  5.1 . The major differences in implementation are described below:

In the instances tested in Sect.  5 , the \(A_i\) matrices encountered were sparse. In the phase retrieval problems we test in this appendix, the \(A_i\) matrices are dense but rank-one. The implementations of CertSDP, CSSDP, and SketchyCGAL are modified to handle such instances.

Phase retrieval instances are likely to contain many dual optimal solutions that may not satisfy strict complementarity. Within CertSDP and CSSDP, we employ the Accelegrad algorithm to approximately solve

Here, \(\lambda _{1+2}(\cdot )\) denotes the sum of the two smallest eigenvalues of a given matrix and is a concave expression in its input. This penalization/regularization encourages solutions \(\gamma \) for which the second eigenvalue of \(I - G^\intercal {{\,\textrm{Diag}\,}}(\gamma )G\) is positive, so that \(A(\gamma )\succ 0\) . We set \({{\textrm{penalty}}} = 10\) .

Recall that in Sect.  5 , we replaced the random sketch in SketchyCGAL with a projection onto a submatrix to reflect the fact that for QMP instances, the goal is to recover the \((n-k)\times k\) top-right submatrix of the SDP optimizer. For the phase retrieval experiments, we employ the random sketch as originally described in [ 75 ].

Numerical results. Due to memory constraints associated with storing \(G\in {\mathbb {R}}^{m\times n}\) , we test instances with size \(n = 30,\, 100,\,300\) . We set the time limit to 50, 500 and 5000 s respectively. The results are summarized in Tables  4 ,  5 and  6 . The average memory usage of the algorithms is plotted in Fig.  4 . We compare the convergence behavior of CertSDP with that of CSSDP and SketchyCGAL on a single instance of each size in Fig.  5 .

Memory usage of different algorithms on our phase retrieval instances as a function of the size n . In this chart, we plot 0.0 MB at 1.0 MB (see Remark  8 for a discussion on measuring memory usage)

Comparison of convergence behavior between CertSDP (Algorithm 2), CSSDP, and SketchyCGAL on our phase retrieval instances. The first, second, and third rows show experiments with \(n=30\) , 100, and 300 respectively

The results for these experiments are qualitatively similar to those of Appendix  5 . We make a few additional observations:

On these phase retrieval instances, the dual suboptimality decreases to \(\approx 10^{-3}\) before CertSDP seems to find a certificate of strict complementarity (see Fig.  5 ). This suggests that the value of \(\mu ^*\) in these instances is relatively small.

CSSDP outperforms SketchyCGAL and also outperforms CertSDP initially. The “crossover” point where CertSDP outperforms CSSDP occurs only after CSSDP is able to produce a primal iterate with squared error \(\approx 10^{-7}\) .

CertSDP seems to suffer from numerical issues for \(n = 300\) and is unable to decrease the primal squared error beyond \(10^{-10}\) . Nonetheless, CertSDP outperforms CSSDP and SketchyCGAL on all instances tested.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Wang, A.L., Kılınç-Karzan, F. Accelerated first-order methods for a class of semidefinite programs. Math. Program. (2024). https://doi.org/10.1007/s10107-024-02073-4

Download citation

Received : 02 June 2022

Accepted : 13 February 2024

Published : 22 March 2024

DOI : https://doi.org/10.1007/s10107-024-02073-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Semidefinite program
• First-order method
• Quadratic matrix program
• Optimal storage
• Find a journal
• Publish with us
• Track your research