My boy also has a little yellow since birth, it's been 13 days now but still the same, doctor told me it's not serious, I can understand how you feel, bless your baby.
My boy is just 7-day old now, in last 2 days, I need to bring him back to STH for blood test, when I saw the blood from his foot, my tears come out! I also worry a lot and could not sleep for 2 days. Luckily after yesterday's blood test, the doctor said the result degree is dropping and baby does not need to go back.
Don't worry too much and trust your doctor.