Summary (TL;DR)
Refactoring means changing code structure without changing observable behavior. Changing a public API interface is not refactoring it is a redesign because it breaks consumers. Internal interface changes that don't affect external callers can be refactoring. The definition depends on perspective: a change is refactoring only if it doesn't alter behavior for its specific consumers.
