Mediator vs OSB

1. Mediator is an internal component in a composite application and can be used to mediate between the components or the component and the outside world. OSB is a standalone full function powerful stateless ESB that is an intermediary between hetrogenous clients and services and is a part of neither of them. 
2. Mediator is primarily targeted to composite developers. OSB is primarily targeted to a system integrator using the console except if advanced programming concepts like split join is used. In that case the optional eclipse IDE can be used by the system integrator.
3. OSB is a fully fledged standalone stateless ESB, and works an intermediary between service consumers.
4. OSB working as a proxy or a differentiated layer.
5. Mediator works as an aggregator or mediator of components within a composite application or by mediating components from external systems
6. Mediator is a replacement to ESB and takes care of the communication brokering within an application.

Mediator

Oracle Mediator facilitates integration between events and services where services invocations and events can be mixed and matched. You can use a mediator component to consume a business event or to receive a service invocation. A mediator component can evaluate routing rules, perform transformations, validate, and either invokes another service or raises another business event. You can use a mediator component to handle returned responses, callbacks, faults, and timeouts. In addition, you can also implement a variety of integration patterns such as service virtualization, publish and subscribe fan-in, and fan-out and various synchronous and asynchronous request response patterns.