COMMUTER ACFT HAS NMAC WITH SMA IN IMC.

Narrative

WE WERE ON THE SHAFF 3 ARR INTO NEWARK; BUT HAD BEEN CLRED TO DEVIATE FOR WX IN OUR DSCNT TO 7000 FT. THE CLRNC WAS 'CLRED TO DEVIATE R AND THEN DIRECT SPARTA (SAX) WHEN ABLE.' WE REACHED 7000 FT AND PROCEEDED DIRECT SAX WITH ZNY. ZNY SAID WE HAD VFR TFC AT 12 O'CLOCK AND 10 MI; 'ALT INDICATES 6000 FT.' WE SELECTED TCASII 10 NM RANGE AND SAW THIS TFC ON TCASII AT 6000 FT AS THE CTLR HAD INDICATED. SINCE THE CLOUD WENT FROM ABOUT 6000 FT TO ABOVE 10000 FT IN THIS AREA AND COVERED ABOUT 60 PERCENT OF THE SKY; WE COULD NOT VISUALLY SEE THE TFC AND STATED TO ZNY THAT WE WERE LOOKING FOR THE TFC. CTR GAVE US A HDOF TO NEWARK APCH WHO ASKED IF WE WERE UP BEFORE WE COULD EVEN CHK IN. I SAID 'YES; AND LOOKING FOR THE TFC ON OUR NOSE AT 6 MI.' THERE WAS NO WAY A VFR ACFT COULD BE AT 6500 FT AND HAVE LEGAL CLOUD SEPARATION; BUT WE SAW ON TCASII THAT THE TFC HAD CLBED TO 6500 FT AND WAS APPARENTLY LEVEL NOW AT ABOUT 4 NM STILL ON OUR NOSE. I SUGGESTED THE CAPT TURN L JUST AS NEWARK APCH GAVE US A 30 DEG L TURN; HOWEVER; THE TCASII NOW INDICATED THAT THIS 'VFR' TFC WAS AGAIN CLBING WITH ONLY 2 MI SEPARATING US. THE TCASII INDICATED 400 FT SEPARATION AND GAVE US AN RA TO CLB. THE CAPT TOOK ABOUT 2 SECONDS TO FIND THE AUTOPLT DISCONNECT BUTTON WHICH SEEMED LIKE AN ETERNITY TO ME; SO I ALSO SAID 'CLB.' WE DID CLB WITH THE 'VFR' TFC PASSING WITHIN 300 FT BELOW US 'IN THE CLOUDS.' WE NEVER VISUALLY SAW THE ACFT. WENT TO 7800 FT; TCASII SAID 'CLR OF CONFLICT' AND WE DSNDED AND INFORMED ATC OF THE RA. LESSONS LEARNED: DON'T TRUST 'VFR' TFC TO STAY VFR. DEFINITELY FOLLOW TCASII GUIDANCE -- IT WORKS! AND WHEN YOU ARE ON AUTOPLT AND GET A TA; I WOULD RECOMMEND EITHER DISCONNECT THE AUTOPLT OR HAVE YOUR THUMB ON IT AND BE PREPARING FOR THE RA TO FOLLOW. ATC WAS EXCELLENT AND DID EVERYTHING THEY COULD; BUT THIS GUY WAS UNPREDICTABLE.