mTW523 - The modified TW523 project
The Modified TW523.
- Intro
-- Introduction
-- A word about safety
-- tools
- Theory
-- Layer 1 the voltages and frequency
-- Layer 2 the half bit, bits, packets and errors.
--- Standard packets
--- Dim/Brights packets
--- Extended packets
-- My intention (theory)
- Hardware
-- TW523
--- Default Operation
--- Modified Operation
- Software
-- PIC
-- PC
-- Future additions
-- Intro
Here's a project that started out as a way to see the bits of X10
traverse the AC and as a way for me to brush up on the electronics
that I really haven't used since I left college in 1987. So I've kept
it relatively simple so students should be able to follow along and
people who haven't done this in a while can say 'Oh, yeah, I remember
that.'.
OK, now I've been playing with X10 going way back to the days of the
brown BSR units. Back then I had the Heath transmitter kit (similar to
X10's CP290) connected to to my Atari 520ST and then my AT&T 3B1 (a
Unix PC). X10 was a mysterious protocol and back then I just wanted
it to work. As the years rolled by I became more interested in how
X10 worked and if I could build an controller to communicate with X10
devices. I read the X10 Tech note and understood how the on and off
commands worked but the description of the other commands left
something to be desired. In addition the TW523 could transmit any
command but couldn't receive all the commands. With X10's CM11A the
TW523 seems like some what of a kludge. Now if only X10 would do
something about it's quality control. The good news is that on the
newsgroup: comp.home.automation, many of the regulars have worked out
to fix things and how to improve the performance.
The motivation behind this is simply that I wanted to see all the bits
and the noise. The current TW523 couldn't do it and the CM11A masks
the protocol with another layer of software. The PowerLinc wasn't
available at the time and I'm not sure how well it works. There have
been reports of problems with the powerlincs. So I went about
modifying the TW523 so it would pass everything. The intelligence
would be added to a microcontroller connected to the outside of the
opto-coupler. This would make it safe for the microcontroller to
operate. I had thought of using a uController inside the TW523 but I
decided against it because of safety when debugging. Remember that I
doing this from home where my lab is made up of inexpensive tools that
I've gathered over the years. I'm currently using Microchip's ICD
(got it during the $99 sale). Having that in the middle of the of a
hot ground would be very bad for the PC to which it was connected to.
[ Need picture of 'hot' ground. ]
- A word about safety.
One of the things to note about this circuit is that it is not
isolated from the live AC. That bears repeating: Warning system ground
is not isolated from AC neutral!!!
I once worked with an engineer whose analog was weak. In his defense
he strengths are digital and assembly language. He could write
assemble like many good programmers can write in high level langauges.
One day while teaching a class on basic electronics he took the probe
of the oscilliscope and stuck the end into a near by AC outlet. In the
process the scope died in a spectacular puff of smoke and noise. The
following day the engineer related his story to me. I in turn
explained to him that what he had done was to short the AC to ground.
I further explained that when ever working with older scopes and AC
that an isolation transformer should always be used. Yes I have one
directly connected to the HP scope I'm currently using. Now this
wasn't to belittle engineers but rather point out that sometime those
who should know may not be ware of the dangers
-- Tools
Tektronics 2230 100 MHz digital storage oscilloscope.
Bitscope.
bread board
[ need picture of bread board ]
- Theory
-- Layer 1 the voltages and frequency.
Theory
X10 was design in Scotland where 50Hz is common
In the US we have 60Hz
X10 basically sends a 120K Hz spurt within 50us of zero crossing of
the AC. The zero crossing is where the AC's voltage is 0.
A data bit is sent on the first half of the AC cycle and it's
compliment is sent on the second half. It doesn't matter if the signal
starts out rising or falling as long as there have been 6 ZC's where
no data has appeared.
Starting with what has been transmitted onto the AC we have the
following: The transmitter transmits a 120KHz pulse starting up to
50uS after the Zero Crossing (ZC), the pulse is 1mS in duration (we
will ignore the other 2 pulses, which correspond with the second and
third phase, for now). An X10 message consists of at least 2 halves.
cEach half is 22 bits long (most messeages, there are exceptions). The
first half (ID) is sent twice, then a pause of 6 ZC's then the second
half (CMD) is sent twice. The transmitter must wait at least 6 ZC's
before starting to transmit. The entire message takes 94 bits to
send, plus wait 6 ZC's before starting, grand total 100 ZC's. Strange
how this corresponds exactly to 50Hz (100 ZC's). The TW523 will send
the signal to the end user device as it starts to see the second half
of the command. I currently don't know what it does if the second
half of the command has bit errors. The original TW523 can not see
Dims, Brights, Extended Commands or Extended Data because none of
these wait for the 6 ZC's between commands.
[ need picture of TW523's proper output]
A single X10 command such as turn device A1 on takes a grand total of
50 cycles. For those in Europe (X10 was invented in Scotland) that
means that each command takes 1 second. For those of us on 60 cycle
AC the each command takes ~0.833 seconds. Here is the break down
(this assumes nobody is sending or in the middle of sending):
1) ( 3) The sending device should wait for 3 cyles with no signal
seen.
2) ( 2) Then send the Start of Header (1110) this takes 2 cycles.
3) ( 9) Then send the first half of the command (Letter code/Number
code) this takes 9 cyles.
4) (11) Repeat steps 2 & 3.
5) ( 3) Pause for 3 cycles.
6) ( 2) Then send the Start of Header (1110) this takes 2 cycles.
7) ( 9) Then send the second half of the command (Letter
code/Function code) this takes 9 cyles.
8) (11) Repeat steps 6 & 7.
Done!
3 + 2 + 9 + 11 + 3 + 2 + 9 + 11 = 50
One can argue that the first 3 cycle pause can be ignored but I don't
recommend it. Also this doesn't take into consideration the Bright/Dim
commands or the Extended Data/Command Commands. These commands don't
have the middle pause between the Letter/Number packet and the
Letter/Function packet. But did you notice that the total is 50
Hz. Yes it takes 1 sec for X10 to send a command to a device. In
Canada and the US it takes 833.33 mS.
-- Layer 2 the half bit, bits, packets and errors.
--- Standard
Definitions:
Half bit = the signal present on half a cycle of the carrier
Should I switch from using the term 'half bit'
[SOH] = Start of Header (1110, first 4 half bit pattern)
Hc = House Code (8 half bits)
Uc = Unit code (10 half bits)
Fc = Function Code (10 half bits)
(P) = Pause (000000, 6 zero half bits)
(P)+ = Pause plus (6 or more zero half bits) mTW523 outputs a at
the 7th zero half bits.
ZC = Zero Crossing, where the AC voltage = 0V
------ --- ---- --------------- ------------------
[ Hc Uc Fc Notes ]
------ --- ---- --------------- ------------------
[ 0000 ] M | 13 | All Units Off | ]
[ 0001 ] E | 5 | All Light On | ]
[ 0010 ] C | 3 | On | ]
[ 0011 ] K | 11 | Off | ]
[ 0100 ] O | 15 | Dim | ]
[ 0101 ] G | 7 | Bright | ]
[ 0110 ] A | 1 | All Light Off | ]
[ 0111 ] I | 9 | Ext Code 1 | (Ext Code) ]
[ 1000 ] N | 14 | Hail Request | ]
[ 1001 ] F | 6 | Hail ACK | ]
[ 1010 ] D | 4 | Ext Code 3 | (Preset Dim (1)) ]
[ 1011 ] L | 12 | Unused | ]
[ 1100 ] P | 16 | Ext Code 2 | (Preset Dim (2)) ]
[ 1101 ] H | 8 | Status On | (Ext Data) ]
[ 1110 ] B | 2 | Status Off | ]
[ 1111 ] J | 10 | Status Req | ]
------ --- ---- --------------- ------------------
--- Dim/Bright
[ C16 C16 COn COn ] or [ C16 COn ] Repeat
V
[ SOH ] [ HC ] [ UC ] [ 01 ] ... [ SOH ] [ HC ] [ CMD ] [ 10 ] ...
[ 1110 ] [ 01 01 10 01 ] [ 10 10 01 01 ] [ 01 ] ... [ 1110 ] [ 01 01 10 01 ] [ 01 01 10 01 ] [ 10 ] ... 000000
[_SOH__] [______C______] [_____16______] [_Nn_] ... [_SOH__] [______C______] [_____On______] [_Fn_]
17 07:20:09 - 11100110100110101001011110011010011010100101000000
17 07:20:09 - (SOH) A.2 (SOH) A.2
17 07:20:09 - 11100110100101011010101110011010010101101010000000
17 07:20:09 - (SOH) A.Off (SOH) A.Off
A 00 is an idle condition (nothing is being sent).
A 01 is a 0 bit (if the SOH has been sent).
A 10 is a 1 bit
And a 11 is can error or the first part of an SOH.
--- Extended
As far as we know (at the time of this writing) only "Extended Code 1"
has a defined frame length which is 31 cycles (62 bits) and is
described as:
* Start Code = 4 bits,
* Housecode = 4 bits,
* Extended code 1 = 5 bits (01111),
* Unit code (device code) = 4 bits,
* Data = 8 bits,
* Command = 8 bits..
# Extended code A[1]
#
#16 00:52:50 - 1110 01101001 0110101010 01101001 10010110.10010110 10011010.01010101 1110 0110100101101010100110100110010110100101101001101001010101000000
# [ SOH ] [ HC ] [ FC ] [ UC ] [ Data ] [ Cmd ]
# 0110 01111 0110 1001.1001 1011.0000
# A Ext 1 1 9 9 B 0
#16 00:52:50 - 1110011010010110101010011010011001011010010110100110100101010111100110100101101010100110100110010110100101101001101001010101000000
The explanation for not having a defined frame length for the other
two is:
"Extended code 2 is variable in length, depending on the type of
message. It has its own separate "attention" marker to separate it
from all other formats.
Extended code 3 has been "assigned" for security but doesn't actually
exist yet so its format has not yet been defined."
- My intention (theory)
My intention for the mTW523 were really simple. I just wanted to see
every half bit that was on the AC. The theory is simple enough, block
the 60 Hz signal while allowing a 120K Hz signal through. Then convert
that 120K Hz signal to a logic 1. The TW523 already does most of this
for you except it blocks half the signal. So I decided to modifiy the
TW523. I got the idea that if I used a transistor as a switch I could
have it on when a signal was present. So I started from there.
- Hardware
-- TW523
--- Default Operation
The TW523 provides a low impedance path for the 120KHz signal and a
high impedance path to the 60Hz via a capacitor (C10). It also limits
the available current via a 100K resistor (R11). It then proceeds
through a isolation transformer and gets clipped by diodes (D3 &
D4). It then monitors the half bits for the pattern 1110 which is the
Start of header (SOH). The next 8 half bits are the house code.
Followed by 10 bits for the unit code or the function code (the last 2
bits determine whether it's a function code (a command) or a unit
code).
An unmodified TW523 can not receive extended codes
The general principle of the the TW523 is to read the 120K Hz signals
and filter only valid X10 packages. It reads in 22 cycles of data and
passes the last 11 cycles of data to the end device. The chip that
X10 uses has been described as an analog shift register
I was a little surprised when I came up with the idea to replace the
X10 IC with a transistor amp design. I wondered, why had no one else
attempted it? The theory is simple enough, a valid X10 signal would
have a 120K Hz for 1.04 mS starting within 50 uS of the zero crossing.
Since I was lazy I realized that a pulse of a certain voltage level
amplified to greater than 5v would look like a TTL logic 1 after it
had passed through the opto isolator. I'd see every bit including
invalid packets (from noise or collsions). Of course it's not perfect
as signals less than nV (1.2V?) pk-pk would not be seen.
--- Modified Operation
- Software
-- PIC
After I got the modified TW523's ouput real nice and pretty (well,
close enough for government work ....) I needed a way to convert the
output to something a computer can deal with. Since I didn't want to
bog down the computer with thet details of bits and zero corssings I
went with a PIC chip. The reason I chose the PIC's was that I had the
Microchip ICD and I had a C compiler for the 14 bit flash chips. I'm
only just getting started with the AVR's so they wouldn't have made a
good choice in this case.
The software in the PIC was much simplier than I thought it would be
once a few decisions were made. The first decision was that the PIC
wouldn't do the translation. The second was that any 1 half bit would
be sent as an ASCII 1 (0x31) and a 0 half bit an ASCII 0 (0x30). A 1
half bit would start the sniffer transmitting and the 7th 0 half bit
would cause the sniffer to tranmit a through the RS232
port. It then stop sending data until the next 1 half bit woke it
up. This also simplified the code as I could now use the RB0 interrupt
to wake the PIC from it's dormancy.
Links:
I've also added the ability to write bits to the AC line. Using the
Programs on the web site, one can send either fix commands or
raw bit string (sent in hex value). I chose to move the
encoding/decoding out to the user program to make things simplier
within the PIC. The PIC simply receives the bytes and sends out each
bit at the zero crossing. This permits not only valid signals but
signals that the standard X10 is not able to interpret, such as your
own custom protocol. It also allows for us to see what happens on bit
errors and packet corruption.
Future releases of could could also monitor the AC for the identical
signal on other phases (such as all 3 phases as recommended in the X10
Tech. Note). I always wondered about that, here in the states the we
use 2 phases in residential homes, the primary phase and an inverted
phase.
[ need picture ]
# --------------------------------------------------------------------------------------------------------------------------------------------------------------------------
#
# Data to send
# D = 13 bytes
# 1 2 3 4 5 6 7 8 9 A B C D
# C16Con | | | | | | | | | | | | | |
# 14 09:37:21 - <1110 0101> <1001 1010> <0101 0111> <1001 0110> <0110 1001> <0101 0000> <0011 1001> <0110 0101> <0110 0110> <1110 0101> <1001 0101> <1001 1000> <0000 0000>
# E 5 9 A 5 7 9 6 6 9 5 0 3 9 6 5 6 6 E 5 9 5 9 8 0 0
# <1110 0101> <1001 1010> <0101 0111> <1001 0110> <0110 1001> <0101 0000> <0011 1001> <0110 0101> <0110 0110> <1110 0101> <1001 0101> <1001 1000> <0000 0000>
# [SOH] [ 01011001] [ 1010010101 ][ SOH ][ 01010110 ][ 1010010101 ] [ Pause ][SOH][ 01011001 ][ 0101100110 ] [SOH][01011001 ] [0101100110 ][ Pause ] [ Extra ]
# [ 0010 ] [ 11000 ] [ 0001 ][ 11000 ] [ 0010 ][ 00101 ] [ 0010 ] [ 00101 ]
# [ C ] [ 16 ] [ C ][ 16 ] [ C ][ On ] [ C ] [ On ]
# --------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Other options would be to add an ADC circuit (many of the PIC now
include an ADC). We could then measure the voltage levels of the
signal. Sending it through the RS232 would be simple as a 0 would
still mean a 0 half bit. But 1 - 255 would indicate a signal level
(divided by 255) as the 1 half bit.
-- PC
I wrote the first version of the PC's app in Perl. Originally It was
written on a Linux box and I used the Curses module and the Device
Serial module. I hope to rewrite it so that it doesn't use the Curses
module. This module may not be available on Windows machines and I'm
sure that more than a few of you are not ready to load one of the free
Unixes.
Now, the first thing we do is to setup the variables that we need then
we setup the serial port (9600, 8, N. 1, no handshaking). In the
curses version we setup our windows and open our log file. In the
while(1) loop we get down to work. Next we setup select and for many
of the non Unix programmers you've probably not seen select(). Select
lets us sit there and wait for something to happen with out polling
the items of interest. This reduces the CPU utilization
dramatically. When select returns we most likely have something to
do. Next we check for what that may be. If it's input from the serial
port we take it and parse it. So far my translator assumes a perfect
string. From what I've seen I really need to fix this as my X10 exists
in a dirty environment. I'll fix that later. After we parse the line
we print it to the appropriate window and the log file. The other item
of concern is the user input. Acceptable inputs are quit, clear a
window or both, and adding a comment to the log.
In the original version of the PIC code I'd send a line of data 1
character at a time and terminate it with a . This worked
great and I'd see all the bits and not worry about the 'dead air'
between commands. But I started to notice that some commands that
should be grouped together were being sent as 2 or more parts. This
meant that there were at least 7 zero crossing with zero half bits. I
couldn't tell how many bits were being sent so I'm now rewritting it
to send all the bits including the idle bits. This means a lot of the
logic for determining an X10 packet is being moved out to the users
application.
- The sending of data
Now being able to see all the bits on the AC is great but I also
wanted to know what happens when a packet gets sent. Especially when
that packet has corruption in the first or second half of the
packet. So I decided that the method for sending X10 would be handled
at a byte level. That is to say that the packets would be made of
bytes sent from the users app and the PIC program would shift them out
onto the AC. This would allow us to send good and bad packets or even
create our own formats. From the users app we would decode things like
A1 on to the appropriate bits but also permit the sending of manual
packets. Flow control was a bit of a problem which puzzled me for a
while. I didn't want to add more logic to use hardware flow
control. This would allow me to possibly use the smaller 12Cxxx series
of PIC's. The reduction in parts would simplify the building of the
device. I also couldn't use software flow control as if I ever decided
to send the analog level of the X10 signal it could be any value
between 0 and 255. Instead I opted for the users app to monitor the
X10 bits and determine how many bytes had been sent.
Links:
You may email me at ncherry@home.net.