My manager is acting strange

I'm at a small quant firm and I started working for a new manager a few months ago. Our relationship has had its ups and downs, but last week it suddenly got really weird. He avoided me throughout the entire week. When I'd come up to ask a question he wouldn't even look at me. Then we had our 1 on 1, he was asking me about specific results of my recent projects, which was a bit unusual. It felt like he was digging for reasons to punish me around the next performance cycle, but I can't be sure about that. He also seemed nervous at the end of it, and when I was talking during team standup too.

This week, I'd say, was a bit less weird, but he ended the 1 on 1 very early. I could understand that on it's own, but combined with other things, it makes me stressed. My performance has been great so far, and it might be that he just doesn't like me and it doesn't impact my reviews that much, but I don't know if I want to keep working in such dynamic, and if I want to have a manager who is on my back waiting for me to make a mistake.

In the past I tried talking to him and asking what am I doing wrong, referring to these non verbal signals. He seemed to have gotten anxious from me asking, said everything is fine and our relationship seemed to have improved after that. I'm afraid that if I ask him again because of what's happening now, he will just avoid me even more and it will make things worse.

I'm just stressed out and would appreciate any advice or perspective I could get. Should I switch teams? Thanks.