B is probably never bluffing with Player A all-in and almost no side pot. But he can be pushing lower flushes. You're getting around 3 : 1 and need to win ~25% to make money calling. That doesn't seem unrealistic, and I'd call.

Remember, he's probably putting you on AA when you 4B. From his point of view, flush draws were not your reason for getting to the river. Hence he has incentive to bet his own flushes, so you won't get to check back your unimproved top sets.