The 63-year-old Italian was told the news on Saturday morning following their 1-0 home defeat in Group F.
The 2004 champions are bottom of the table with just a single point from four matches and appear unlikely to reach the 2016 Finals in France.
However, Ranieri will remain in charge until after Tuesday's friendly international against Serbia.
A statement from the Greece Federation read: "There was no other option than to terminate the contract."
Federation President Giorgos Sarris said in the statement: "Following the devastating result for the national team, I take full responsibility for the unfortunate choice of the manager."
The federation deny that a successor has already been found, with Greece not playing another competitive fixture until they take on Hungary in March next year.
Speaking after the loss to the Faroes, Ranieri said: "I have no words to explain what happened.
"We all wanted to record our first victory in the group. But the Faroes deserved to win as they created more chances to score. Only hard work can get us out of this situation.
"When the qualifiers started nobody believed we would find ourselves in this situation. We were wrong in believing that was a straightforward group for us.
"All defeats hurt equally but we are last in our group because that is what we deserve."
Ranieri only took over as head coach in July, following Greece's trip to the World Cup finals in Brazil where they reached the knock-out stages.