The effects of the vacuum stress tensor fluctuations of a conformally invariant field, such as the electromagnetic field, in inflationary cosmology will be treated. Normally, one would expect these fluctuations to produce small effects. However, it will be argued that this need not be the case. A model in which a single scalar inflaton is coupled to the conformal field stress tensor will be considered. It will be shown that the fluctuations of the inflaton due to the stress tensor fluctuations can be proportional to a power of the scale factor increase during inflation. This effect can be interpreted as placing a constraint on the duration of inflation. However, this interpretation requires a discussion of the transplanckian problem.