Elevators are one of the most interesting and difficult challenges in civil engineering! They are the prime limiting factor in tower design, and the logic to run them efficiently is very complex. It must anticipate elevator usage demands from all floors, between all elevators, in both directions (up and down), based on the time of the day and even the day of the week. One thing that particularly contributes to people's frustration that "the elevators are ignoring them" is that they can't see other elevator requests in action. Think of it this way: If your building has seven floors, that means 42 point-to-point trips are possible at any one time. But there are only three elevators, so if more than three trips are on the logic board at once, either the extra trips will have to be queued (which is never a first resort) or trips will have to be stacked. Then you get into thousands of potential multi-point trip combinations in a single elevator run. Elevator efficiency can degrade spectacularly badly if the logic is inferior; you've never seen that in action, but just imagine it. Depending on the number of passengers, wait times could easily stretch past the ten-minute mark and toward half an hour.
There are two things you can count on: 1) An elevator will never reverse direction until it has reached the farthest floor in that direction at which service has been requested or the logic itself has told the elevator to go. 2) People inside the elevator always have priority over people still waiting to use the elevator.
One huge step architects and civil engineers can take to improve elevator efficiency is to have as few exit levels as possible--preferably only one, unless we're talking about much taller towers. If a building is on a hill and has street entrances on two levels, that severely degrades the elevator efficiency. If there's a parking garage with multiple parking (i.e., exit) levels in addition to the main level(s), it gets even worse--which is one reason that parking garages often have their own elevators. I am firm advocate that elevators in towers should only have one main exit level, with more added only as absolutely necessary for service and handicap access. (This doesn't count double-deck elevators, which by definition have two exit levels.) All other traffic should be aggregated to or distributed from the main elevator level by use of escalators (for malls, large lobbies, etc.) or secondary elevators (for parking garages, handicap use, and service access). The next best measure is to restrict the main floors at which elevators will stop (which only works well when there are enough elevators and traffic and exit-level spreads for the traffic to be evenly apportioned), so that, for instance, Elevator A will only go to Level 1 while Elevator B will only go to Level B (both elevators will stop at all non-exist levels). In some scenarios, it's the other way around: A given elevator won't serve the even-numbered non-exist floors, and its counterpart the odd-numbered ones, etc.; this has the effect of reducing stops, which are the major source of elevator transit delay. However, either scheme (but especially the former) tends to be confusing to passengers, who respond with characteristic annoyance. If it were my tower, annoyed people would not be allowed to use the elevator. (They'd get to use the Stairway of Surprise.) Beyond these measures, the best way to move people with elevators is to make the elevators faster, the stops shorter, and the bank larger. I had some friends who lived in a tower with almost 30 floors Downtown, and only two elevators. Often enough, one of those elevators would be set aside in midday for people moving in or moving out. Yet service was almost always hearty; there were only a couple of instances that I had to wait more than a minute for the elevator. That's because they were friggin' fast, and had excellent logic. I tried watching a counterpart building in the development, whose elevator traffic is visible from the street, and I was amazed at some of the patterns I saw. Among other things, I realized that the elevator holding floors didn't care which elevator was stationed there. Sometimes it would be Elevator A, and sometimes Elevator B. That's pretty cool. Simpler elevator logic wouldn't be able to handle a paradigm shift like that. Can you tell I like this subject...