In a large stock pot, add whole red potatoes. Top with water, completely covering the potatoes (water should go 1 inch above potatoes) and season with salt. Bring to a boil and cook potatoes until easily pierced through with a fork, about 15 minutes.
Drain potatoes in a colander and allow to cool to room temperature.
Meanwhile, in a bowl add diced celery, diced onion, mayonnaise, mustard, dill, salt, and pepper. Squeeze fresh lemon juice into bowl. Whisk to combine.
When potatoes have cooled to room temperature, slice into halves or quarters (depending on the size of your potatoes - make the pieces bite sized and easy to eat).
In a large bowl, add potatoes and dressing. Toss until potatoes are evenly coated. Refrigerate at least one hour to allow flavors to meld.
Potato salad can be made a day ahead and stored in a tightly sealed container in the refrigerator until ready to serve.
Leftover potato salad will keep for up to 5 days in a tightly sealed container in the refrigerator.
Half or all of the mayonnaise can be substituted with plain Greek yogurt to lower calories.