Broadcasting over a local network ? 
Author Message
 Broadcasting over a local network ?

Hi,
I want to inform other users of the local network about changes on the
database. I could collect the IP-Addresses of each user by sending
individually messages but I thought that it must be simplier by
broadcasting. I spent a lot of time on it but, it is still darkness
all around.
All these Socket's stuff is very interesting but also very complicated
( for me ).
Does anybody have experiences on this item ?
Regards
Jon


Sun, 30 Oct 2005 13:13:27 GMT  
 Broadcasting over a local network ?

Quote:

> Hi,
> I want to inform other users of the local network about changes on the
> database. I could collect the IP-Addresses of each user by sending
> individually messages but I thought that it must be simplier by
> broadcasting. I spent a lot of time on it but, it is still darkness
> all around.
> All these Socket's stuff is very interesting but also very complicated
> ( for me ).
> Does anybody have experiences on this item ?
> Regards
> Jon

Have you tried looking at remote Smalltalk?
http://www.smalltalking.net/Goodies/Dolphin/


Sun, 30 Oct 2005 18:17:30 GMT  
 Broadcasting over a local network ?

Quote:

> I want to inform other users of the local network about changes on the
> database. I could collect the IP-Addresses of each user by sending
> individually messages but I thought that it must be simplier by
> broadcasting. I spent a lot of time on it but, it is still darkness
> all around.

If you want to broadcast the messages reliably, then I think you'll probably
find it easier to hold a TCP/IP session open to each client and send the
notifications to each as a separate message.

If you are happy with potential for missed notifications then you could use
UDP/IP (which sends single "datagrams" rather than holding a reliable
connection open).  Dmitry Zamotkin posted an extension to allow the Dolphin
sockets stuff to talk UDP; it's in a post dated 2001/9/14, titled "UPD Socket"
and you should be able to find it in Ian's archive or possibly via Google.

Your needs might best be met by IP mutlicasting, but I know almost nothing
about that, and nothing whatever about how to do it with Dolphin's networking
support.

So, if *I* were doing this, I'd use a central server, the "database", and each
client that was interested in observing updates would connect to that server.
The server would send a separate message to each connected client whenever the
database changed.  If you need more guidance on the ways you can set up such a
server, and how to connect to it, then do ask again.

    -- chris



Sun, 30 Oct 2005 19:16:46 GMT  
 Broadcasting over a local network ?

Quote:

> So, if *I* were doing this, I'd use a central server, the "database", and each
> client that was interested in observing updates would connect to that server.
> The server would send a separate message to each connected client whenever the
> database changed.  If you need more guidance on the ways you can set up such a
> server, and how to connect to it, then do ask again.

Chris,

I'm working on an application that want's to do something along this line. I'm a
bit of a database novice, and so aren't aware of any features might support this
directly. I was thinking of using a table in the database as sort of a change log
/ notification queue. Each connected client would need to periodically poll that
table with a query. It sounds like maybe you have something else in mind? I'm
using MySQL through odbc at the moment, if that's relevant.

thanks,
-Bill

-------------------------------------------

Shoshana Technologies
100 West Joy Road, Ann Arbor, MI 48105  USA



Sun, 30 Oct 2005 23:56:27 GMT  
 Broadcasting over a local network ?
Bill,

Quote:

> > So, if *I* were doing this, I'd use a central server, the "database",
and each
> > client that was interested in observing updates would connect to that
server.
> > The server would send a separate message to each connected client
whenever the
> > database changed.  If you need more guidance on the ways you can set up
such a
> > server, and how to connect to it, then do ask again.

> I'm working on an application that want's to do something along this line.
I'm a
> bit of a database novice, and so aren't aware of any features might
support this
> directly. I was thinking of using a table in the database as sort of a
change log
> / notification queue. Each connected client would need to periodically
poll that
> table with a query. It sounds like maybe you have something else in mind?

There are people who will insist on doing this kind of thing using database
features.  Borland had it working particularly well quite some time ago.  I
think you'll find that Chris is recommending that you write software that
has clients take a big gulp at the beginning, then notifications to keep
them up to date based on messages/triggers from the server.  The server
would be the only thing that actually touches the database.  FWIW, I agree
with that recommendation.

The most difficult thing about sockets programming is error handling, and
since you will almost certainly want to use multiple threads to make it
work, synchronization issues are a close second.

Quote:
> I'm
> using MySQL through odbc at the moment, if that's relevant.

Good choice, I think.  I've been using MySQL and MyODBC on test machines,
and one entering-production box.  I tried converting my cash cow, and it
stopped giving milk - in fact, it contracted mad computer disease :)  It's
back with an Access database and ODBC until I can figure out what went
wrong.  I'm too busy right now to track it down, but I'm suspicious that I
simply have it configured incorrectly.  Some (but not all) database access
caused the mysqld-nt service to jump to 99% (and then some I think) CPU
utilization.  Please let me know if you've ever heard of a similar
complaint.

Have a good one,

Bill

--
Wilhelm K. Schwab, Ph.D.



Mon, 31 Oct 2005 01:48:26 GMT  
 Broadcasting over a local network ?

Quote:

> There are people who will insist on doing this kind of thing using database
> features.  Borland had it working particularly well quite some time ago.  I
> think you'll find that Chris is recommending that you write software that
> has clients take a big gulp at the beginning, then notifications to keep
> them up to date based on messages/triggers from the server.  The server
> would be the only thing that actually touches the database.  FWIW, I agree
> with that recommendation.

This sounds somewhat like what I'd like to evolve to. Right now (as far as the
application is concerned) only MySQL is running on the server, with a Dolphin
app on the client connected to it over the Internet. I'd like to migrate most of
the application to the server, leaving only the top UI layer running in the
Dolphin app on the client. But that looks like an involved undertaking, so I'm
also looking at how to use just the database for change notifications, as a
stopgap measure.

In the long run, I still like using Dolphin for the clients. Being able to
produce nice looking applications, the ease of interfacing to Windows functions
when needed, reasonably sized downloads, etc. It's all good. For the server side
of things, it's more of a conundrum. I _really_ don't want to change the server
from Linux to Windows, so that leaves Dolphin out of the picture. So I'm
considering VisualWorks and/or GemStone for the server. Though working in
multiple environments is always more of a challenge. Course that keeps things
from getting boring. :-)regards,
-Bill

-------------------------------------------

Shoshana Technologies
100 West Joy Road, Ann Arbor, MI 48105  USA



Mon, 31 Oct 2005 10:33:44 GMT  
 Broadcasting over a local network ?

Quote:

> Some (but not all) database access
> caused the mysqld-nt service to jump to 99% (and then some I think) CPU
> utilization.  Please let me know if you've ever heard of a similar
> complaint.

Let me pass one on first-hand! What I found caused the 99%+ CPU utilization was
triggered by doing a "drop table". As long as I avoided that, I didn't have a
problem. It was coming up quite a bit in putting together SUnit tests on the DB.
:-(  I resorted to dropping and re-creating the entire test database rather that
dropping tables in a existing one. That at least minimized the occurrences of
the problem to real database schema changes.

An example sequence from the the mysql.err file might go like this:
    030128 20:26:01  InnoDB: Warning: MySQL is trying to drop table
test/#sql2-3fc-224
    InnoDB: though there are still open handles to it.
    InnoDB: Adding the table to the background drop queue.
    030128 20:40:46  MySql: Normal shutdown
    030128 20:40:46  InnoDB: Starting shutdown...
    030128 20:40:47  InnoDB: Dropped table test/#sql2-3fc-224 in background drop
queue.
    030128 20:40:50  InnoDB: Shutdown completed
    030128 20:40:50  MySql: Shutdown Complete

Once the table was added to the background drop queue, the CPU would peg, and
that would be it. I found that if one exhibited enough patience to wait for
menus to respond and shut down the MySQL server normally, that the drops would
actually be processed during shutdown, as shown in the log above. At one time, I
tried to figure out what it might mean that there were "still open handles" to
it, but didn't have any luck.

This was with MySQL 3.23.55 on a Win2k box (connecting with MyODBC 2.50.39).
I've wondered whether MySQL 4.0 has fixed the problem, but haven't tried it. My
real solution was <drum roll> to switch to running MySQL on a Linux server,
where the issue just doesn't occur.

HTH,
-Bill

-------------------------------------------

Shoshana Technologies
100 West Joy Road, Ann Arbor, MI 48105  USA



Mon, 31 Oct 2005 10:37:17 GMT  
 Broadcasting over a local network ?
Bill,

Quote:
> Once the table was added to the background drop queue, the CPU would peg,

How did you identify the event?  Is it simply the last thing logged before
the problem occured, or were you able to see it happen?

Quote:
> and
> that would be it. I found that if one exhibited enough patience to wait
for
> menus to respond and shut down the MySQL server normally, that the drops
would
> actually be processed during shutdown, as shown in the log above.

How long did that take to regain control?  Might it help to have a stop
button on the process in plain view before attempting to reproduce the
problem?

Quote:
> At one time, I
> tried to figure out what it might mean that there were "still open
handles" to
> it, but didn't have any luck.

Did you report it?  It might make sense to the developers.  One possible
scenario would be that the only thread that can run polls the queue - of
course nothing like that ever happened to any of my code :)  Humor aside, it
might be just the observation they need to find a synchronization bug, so if
you are confident of your findings, I urge you to submit a report to MySQL
AB.

Quote:
> This was with MySQL 3.23.55 on a Win2k box (connecting with MyODBC
2.50.39).
> I've wondered whether MySQL 4.0 has fixed the problem, but haven't tried

it.

I'm reasonably certain that I have 4.0 on the offending box, but I'll check.

Quote:
> My
> real solution was <drum roll> to switch to running MySQL on a Linux
server,
> where the issue just doesn't occur.

Are you using ODBC over a network to get to it?  That might not be a bad
solution for us.  The only problem (and I think it is is possible) is that
I'd want the database traffic encrypted.

Still, I'm left with the questions of what clobbered the machine, and why
does it apparently not happen on the test machines?  It's ironic that your
problems centered around schema changes.  Aside from the usual
fast/stable/free benefits, and MySQL's being a network aware server, one of
the things that has pulled me into it is the ability to control the schema
from my code; Access' SQL falls just a little short in a couple of areas.  A
silly example is the ability to re-order fields the way I want; MySQL can do
it, and Access apparently cannot.  One could argue that it shouldn't matter,
but the fact is that it does matter to one of my systems.  I could maintain
a separate field order, but it's much nicer to simply get it from the
database.  Finally, if only thanks to Paul DuBois, is very well documented.

Thanks for the info!!!

Bill

--
Wilhelm K. Schwab, Ph.D.



Mon, 31 Oct 2005 20:38:08 GMT  
 Broadcasting over a local network ?

Quote:


> > Once the table was added to the background drop queue, the CPU would peg,

> How did you identify the event?  Is it simply the last thing logged before
> the problem occured, or were you able to see it happen?

The problem with dropping tables was the only issue raised in the mysql.err log
file. (Besides starting and stopping of the server). And I just noticed the
1-to-1 correspondence between that appearing in the error log and the 100% CPU
usage.

Quote:
> > and
> > that would be it. I found that if one exhibited enough patience to wait
> for
> > menus to respond and shut down the MySQL server normally, that the drops
> would
> > actually be processed during shutdown, as shown in the log above.

> How long did that take to regain control?  Might it help to have a stop
> button on the process in plain view before attempting to reproduce the
> problem?

I had WinMySQLAdmin running, so to stop the server I'd click on its icon in the
system tray, wait a fair number of seconds for the menu to appear, go to a
submenu, waiting a fair number of seconds again, and then selecting the stop the
service item, once more waiting a fair number of seconds for it to respond.

I did some more playing with the problem today. Your suggestion was worthwhile.
I had the properties dialog from the control panel showing for MySQL server.
Then it was one simple click on the stop button. Which incidentally took about
20 seconds to shutdown the service, where normally it takes 1 or 2 seconds.

Quote:
> > At one time, I
> > tried to figure out what it might mean that there were "still open
> handles" to
> > it, but didn't have any luck.

> Did you report it?  It might make sense to the developers.  One possible
> scenario would be that the only thread that can run polls the queue - of
> course nothing like that ever happened to any of my code :)  Humor aside, it
> might be just the observation they need to find a synchronization bug, so if
> you are confident of your findings, I urge you to submit a report to MySQL
> AB.

I fired up the Windows MySQL server for the first time in months so that I could
play with the problem again. At first I couldn't seem to reproduce it as simply
as I recalled. I finally worked out that there must be something in the way that
ReStore communicates through MyODBC that rubs the Windows MySQL server the wrong
way. Once I've connected to the db with ReStore and either written something to
a table or done a #synchronizeAllClasses, it's primed to have a problem. I can
disconnect ReStore and even shutdown the Dolphin client application completely.
Of course once I disconnect, the "mysqladmin process" command shows the client
isn't connected at that point. But the server's still ready to blow up. If I
drop one of the tables that had been touched by ReStore, even just by using the
MySQL command line to do it (and not through MyODBC), the problem with needing
to defer the drop gets logged. And then it's about 10 seconds later that the CPU
pegs.

Quote:
> > This was with MySQL 3.23.55 on a Win2k box (connecting with MyODBC
> 2.50.39).
> > I've wondered whether MySQL 4.0 has fixed the problem, but haven't tried
> it.

> I'm reasonably certain that I have 4.0 on the offending box, but I'll check.

Since Dolphin and ReStore aren't exactly standard tools for the MySQL folks, it
would no doubt involve looking into the debug log of MyODBC to see what sequence
of things might set it off. I think I'll wait until I get a chance to install
4.0 and the current MyODBC and see if there's still a problem before reporting
it though.

You're not using ReStore by any chance are you? Because it's doing something
through the MyODBC interface that I haven't been able to reproduce simply using
DBConnection. I'm wondering if you're having the same problem that I ran into,
or perhaps what you're doing is just uncovering the same underlying problem with
the Window's version of MySQL server? As I said before, the same sequence fed to
a Linux MySQL server doesn't cause any problem.

Quote:
> > My
> > real solution was <drum roll> to switch to running MySQL on a Linux
> server,
> > where the issue just doesn't occur.

> Are you using ODBC over a network to get to it?  That might not be a bad
> solution for us.  The only problem (and I think it is is possible) is that
> I'd want the database traffic encrypted.

Yes. Even the Window's MySQL server tended to be on a different machine than the
Dolphin client. Found that it's real simple to get it going just by entering in
the host name, or even a full domain name into the configuration for the ODBC
data source in the control panel. With that it's easy to connect to MySQL
anywhere on the LAN or even the Internet. Though in the latter case connection
speed and probably even more important the latency become issues.

Encrypting the database traffic is something I'd be interested in as well. I
haven't looked into it yet. If you come up with anything, let me know.

regards,
-Bill

-------------------------------------------

Shoshana Technologies
100 West Joy Road, Ann Arbor, MI 48105  USA



Tue, 01 Nov 2005 11:22:26 GMT  
 Broadcasting over a local network ?


Quote:

> > There are people who will insist on doing this kind of thing using database
> > features.  Borland had it working particularly well quite some time ago.  I
> > think you'll find that Chris is recommending that you write software that
> > has clients take a big gulp at the beginning, then notifications to keep
> > them up to date based on messages/triggers from the server.  The server
> > would be the only thing that actually touches the database.  FWIW, I agree
> > with that recommendation.

> This sounds somewhat like what I'd like to evolve to. Right now (as far as
> the
> application is concerned) only MySQL is running on the server, with a Dolphin
> app on the client connected to it over the Internet. I'd like to migrate most
> of
> the application to the server, leaving only the top UI layer running in the
> Dolphin app on the client. But that looks like an involved undertaking, so
> I'm
> also looking at how to use just the database for change notifications, as a
> stopgap measure.

> In the long run, I still like using Dolphin for the clients. Being able to
> produce nice looking applications, the ease of interfacing to Windows
> functions
> when needed, reasonably sized downloads, etc. It's all good. For the server
> side
> of things, it's more of a conundrum. I _really_ don't want to change the
> server
> from Linux to Windows, so that leaves Dolphin out of the picture. So I'm
> considering VisualWorks and/or GemStone for the server. Though working in
> multiple environments is always more of a challenge. Course that keeps things
> from getting boring. :-)regards,
> -Bill

> -------------------------------------------

> Shoshana Technologies
> 100 West Joy Road, Ann Arbor, MI 48105  USA

What about making a server (using Dolphin) that runs on Windows that
talks to the database that runs on linux?


Tue, 01 Nov 2005 15:20:59 GMT  
 Broadcasting over a local network ?

Quote:

> I'm working on an application that want's to do something along this
> line. I'm a bit of a database novice, and so aren't aware of any
> features might support this directly. I was thinking of using a table
> in the database as sort of a change log / notification queue. Each
> connected client would need to periodically poll that table with a
> query. It sounds like maybe you have something else in mind? I'm
> using MySQL through odbc at the moment, if that's relevant.

I'm afraid that I don't have a good solution.  I was assuming that Jon already
had a way of telling when/what updates were made to the "database" (in quotes
since I didn't get the impression that he was necessarily using an
off-the-shelf SQL db), and that the only issue was how to distribute the
notifications.

It may well be that MySQL has the means for clients to Observe (in the sense of
the Observer pattern) changes to the db, but I don't know how to do it.  In any
case, I don't think (though this is *not* something I know much about) that
standard SQL or ODBC provides a way to do it.  If there are such facilities
then you'd probably have to use a direct API to get at them.

<random noodlings>
Without knowing MySQL, it seems that your approach is as viable as anything
that doesn't require direct access to MySQL-specific APIs.  I'd be inclined to
use triggers internally to add rows to the "this has changed" table.  Depending
on the application profile you might find it better for the clients to
broadcast a notification "something has probably changed, check the DB" to each
other (possibly via a central server) when they commit changes to the DB,
rather than polling all the time.  You might even use a hybrid with polling at
low frequency (once a minute, once a day, whatever) supplemented by broadcast
notifications to improve responsiveness.

Come to think of it, you might be able to use server-side Perl/Python/whatever
to hook the changes and broadcast notifications.  If that's possible at all,
then it'd probably be pretty easy.
<random noodlings>

    -- chris



Tue, 01 Nov 2005 20:41:53 GMT  
 Broadcasting over a local network ?

Quote:



> > In the long run, I still like using Dolphin for the clients. Being able to
> > produce nice looking applications, the ease of interfacing to Windows
> > functions
> > when needed, reasonably sized downloads, etc. It's all good. For the server
> > side
> > of things, it's more of a conundrum. I _really_ don't want to change the
> > server
> > from Linux to Windows, so that leaves Dolphin out of the picture. So I'm
> > considering VisualWorks and/or GemStone for the server. Though working in
> > multiple environments is always more of a challenge. Course that keeps things
> > from getting boring. :-)

> What about making a server (using Dolphin) that runs on Windows that
> talks to the database that runs on linux?

Certainly something to consider. Still, my confidence in using Windows as a server
platform is nowhere near as strong as that of using Linux. And in that senario, it
would still be acting as a critical server component, even if it wasn't handling
the database. Plus, since we don't have the bandwidth here, seems it would require
a minimum of two machines at a collocation facility, even to get started. In the
long run, it would be nicer to be supporting a more uniform mix of machines on any
(potential) server farm. Thanks for the idea, though.

regards,
-Bill

-------------------------------------------

Shoshana Technologies
100 West Joy Road, Ann Arbor, MI 48105  USA



Wed, 02 Nov 2005 06:14:46 GMT  
 
 [ 21 post ]  Go to page: [1] [2]

 Relevant Pages 

1. Scanning for local servers, getting a broadcast address

2. Network Broadcast messages in Windows

3. How to get a MAC-Address from a PC in a local network

4. Copy file from network drive to local drive?

5. Novell over Network faster than 2k Local

6. Local Printing Err while on network

7. Using audio across a local network

8. core dump when receiving multicast packets on a local network

9. Local Procedure vs local Routine

10. Sequence local = local variable?

11. local variable and local variable in block behave differently

12. writing to locals (was RE: execfile: NameError exception thrown for things in locals())

 

 
Powered by phpBB® Forum Software