The Bible contains many promises about prayer, for instance:
Mark 11:22-24
22 And Jesus answering saith unto them, Have faith in God.
23 For verily I say unto you, That whosoever shall say unto this mountain, Be thou removed, and be thou cast into the sea; and shall not doubt in his heart, but shall believe that those things which he saith shall come to pass; he shall have whatsoever he saith.
24 Therefore I say unto you, What things soever ye desire, when ye pray, believe that ye receive them, and ye shall have them.
Matthew 21:21-22
21 Jesus answered and said unto them, Verily I say unto you, If ye have faith, and doubt not, ye shall not only do this which is done to the fig tree, but also if ye shall say unto this mountain, Be thou removed, and be thou cast into the sea; it shall be done.
22 And all things, whatsoever ye shall ask in prayer, believing, ye shall receive.
Similar promises can be found in 1 John 5:14-15, John 14:13-14, 15:16, 16:23-24, Matthew 7:7-11, Luke 11:5-13, and Psalm 50:15, 86:7. Some of these, like the passages printed above, seem to have no caveats whatsoever, in their context.
Yet, the fact is, Christians ask for many things that they don't receive. Why?
Firstly, before getting into specific hindrances to prayer that are mentioned in Scripture, it should be noted that the prayer request not being answered if it is not in God's will should be an obvious implied caveat in all passages pertaining to prayer promises, assuming one understands the nature of God. Should God answer a prayer that will be ultimately detrimental to the person praying, or others? If something is outside of God's will, it is less than perfect, and so rightfully should not be answered. In this case, a "no" is the best possible answer to receive. Having a robust knowledge of the Word of God is the best way to get an idea as to whether something is in God's will, however, as we see in 2 Corinthians 12:7-9, God's will for individuals may be specific to them.
Now to discuss passages in Scripture which give more insight into why a prayer may not be answered.
There are many passages which prominently mention hindrances to prayers being answered, and those hindrances are:
By looking at the above, we can make two broad categories of hindrances - moral failures by the person praying, or the fulfillment of the prayer not being in God's will.
Moral failure is inevitable if you are a Christian (1 John 1:8), and God has different expectations for different individuals (2 Corinthians 8:12, Mark 12:41-44, Luke 12:48, 21:1-4), depending on a variety of factors specific to them. You do not have to be perfect to get a prayer answered - otherwise, no prayer would ever be answered - but you must live to whatever reasonable standard God expects of you at this point in your Christian walk.
The prayer not being answered if it is not in the will of God - as mentioned in the introduction - should be seen as an obvious implied caveat to all prayer promises. We should be thankful that God would refuse to grant a petition that would be less than His best will for our lives.
So, why pray if the only things that will be answered are things that are in the will of God anyway? Because the Bible is very clear, in almost all of these promises, that there are things that you will get if you pray, which you would not have gotten otherwise (James 4:2, etc.). Prayer does make a difference, but it has some parameters. Do not let that discourage you.
I do not claim to have all the answers as to why specific prayers may not be answered. Sometimes, unanswered prayer puzzles me as much as it does anyone. There is no recourse beyond humbly knowing that God is in Heaven, and sees the end from the beginning, and therefore, knows better than anyone how best to answer any prayer.