Yes, I would be worried because at 5'9 an 160lb, you were at a normal BMI of 23.6.
At 5'9 and 184lb you are overweight with a BMI of 27.2
The calculator I used was: https://www.nhlbi.nih.gov/health/educational/lose_wt/BMI/bmicalc.htm
The question now is: Why?
Your doctor may not be to concerned and this can fall under several reasons such as: Did not read your history chart, did not have enough history with you, or just feel compelled.
This means you need to take it into your own hands. Have you had any different patterns or daily habits from last year?
Has your diet change?
What is your daily routine?
Do you incorporate healthy habits such as gym, avoiding unhealthy foods, skipping meals or such?
If you are not sure, I suggest starting a journal recording daily events such as: When you wake up, what you eat, what is the dosage/weight/serving, where you dine at (if you eat out), when you go to sleep, any medications taken, etc.
After a week you should have enough data to pour through to see some observations that may be linked to the weight gain.
This is assuming the weight gain is from lifestyle and diet intake.