The observer dependence of the quasi-local energy (QLE) and momentum in the Schwarzschild geometry is illustrated. Using the Brown-York prescription, the QLE for families of non-geodesic and geodesic observers penetrating the event horizon are obtained. The QLE of a static, non-geodesic observer is interpreted in terms of a work required to construct the Schwarzschild geometry, whereas the Brown-York QLE of a radially geodesic observer freely-falling from infinity is shown to vanish. Results are compared with the Liu-Yau quasi-local energy in the same settings. Finally, a simple relation for the dynamics of the Brown-York quasi-local momentum density for a geodesic observer field is noted.