I'm not sure when the building ended up being referred to as "the church", but it is the place where the church meets. As we gather in Jesus' name and then carry out Jesus' mission then we are The Church. We the people of God are the church in this place and at this time, but the church of Jesus isn't limited to a time and place. The church of Jesus is the church throughout all time and in all places.
Before Jesus ascended into heaven he gave his disciples one last instruction "Go therefore and make disciples of all nations, baptizing them in the name of the Father and of the Son and of the Holy Spirit, and teaching them to obey everything that I have commanded you. And remember, I am with you always, to the end of the age." Matthew 28:19-20. This is known as the "Great Commission". It is what Jesus sent the church to do, but somewhere along the line we forgot. When I was a teenager we had a saying, "Jesus called us to be fishers of men and not keepers of the aquarium." This has been forgotten and so many congregations practice the latter instead of the former. They even set a standard wishing to keep out those who aren't like them.
Jesus said to teach "them", that is those who are becoming disciples, to obey that which he commanded. In the gospel of John Jesus tells his disciples "I give you a new commandment, that you love one another. Just as I have loved you, you should love one another." John 13:34. We are to go and make disciples teaching them to love as Jesus loved. When we do this we truly are the Church.