Whisk flour and salt together in medium bowl; set aside.
In stand mixer fitted with paddle attachment, beat butter, sugar, and vanilla at medium speed until light and fluffy, about 3 minutes, scraping down sides and bottom of bowl with rubber spatula. Reduce speed to low and add flour mixture, mixing until incorporated. Wrap tightly in the plastic wrap. Refrigerate for at least 1 hour.
Heat oven to 325°. Line a baking sheet with parchment paper.
Roll a heaping tablespoon of dough into a 1½-inch ball, roll the ball in the sprinkles, and place it on the prepared baking sheet. Repeat with remaining dough, spacing balls about 2 inches apart. Using your thumb, make an indentation in the center of each cookie. Bake for 10 minutes, or until slightly firm. Remove the cookie sheets from the oven. Using your thumb, press into the cookies again. Return the cookies to the oven until golden brown, 8 to 13 minutes. Remove cookies to a wire rack and allow to cool completely before icing.
To make the icing, whisk milk and powdered sugar together in a small bowl and add food coloring. Stir to combine and add more milk (up to 2 teaspoons more) to get a smooth and spoonable, but not runny, consistency. Fill centers with icing. Let set overnight. Cookies can be stored in airtight containers up to 1 week.
