Change the hostname of the solaris 10 with out reboot

uname -S <new host name>

This change will exist until the next reboot. 
If you want to permanently change the hostname you need to modify the following files:

Assuming that you have allowed the operating system to configure the primary interface for you, there are three (3) files you must change:

/etc/hosts
/etc/nodename
/etc/hostname.<interface>

<interface> will be the primary ethernet interface for the system. Some examples (depending on your architecture) will be bge0, hme0, e1000g0, eri0, qfe0. Here's an ad-free answer for future reference.

Also change the savecore directory to reflect the new hostname in /etc/dumpadm.conf, and you may also need to edit /etc/inet/ipnodes, depending on the exact version of Solaris.