While the Fantastic Four and X-Men characters have shared the same comics pages with the Avengers countless times, they belong to different universes when it comes to live-action media. This is because their film rights belong to separate studios. While 20th Century Fox owns the Fantastic Four and X-Men, Marvel Studios owns the Avengers and a wide range of Marvel Comics characters.
Sony Pictures owned the film rights to Spider-Man for decades, but they eventually managed to reach an agreement with Marvel about sharing the character and bringing him to the Marvel Cinematic Universe - the world of Iron Man, Captain America, Hulk, and numerous other superheroes. Since that deal took place, we’ve been hoping that something similar would happen between Fox and Marvel.
We’ve heard the Russo Brothers say that they "would love to see" Deadpool in the MCU, Simon Kinberg say that it "would be fun" to do an X-Men/Avengers crossover, and Hugh Jackman say that such a crossover is "possible". We’ve also heard rumors about Fox and Marvel teaming up, and we’ve even seen the two collaborate on a small screen project, the upcoming FX series Legion.
However, it appears that we should no longer expect the Fantastic Four and X-Men to be part of the MCU, as Kevin Feige has closed the door on that. When Variety asked him about the possibility of Fox and Marvel partnering to bring the former's characters to the MCU, the Marvel President responded:
It's an impossibility at this juncture. We certainly have enough films to keep us busy for a number of lifetimes.
So, at this point, Fox and Marvel may have developed a friendly enough relationship to work together on Legion or swap character rights for Deadpool and Guardians of the Galaxy Vol. 2, but not enough to join forces to have the mutants share a screen with the Earth's Mightiest Heroes.