Java programmers here?

Discussion:

Paul G. Allen

2017-08-09 18:11:51 UTC

Anyone here an experienced J2EE engineer (I know there used to be a
few, but I've been semi-lurking for a while)?

I have an issue that I can't figure out. I'm told it can be done, I
found a link through Google with an example, and then I promptly lost
it (and can't find it again - I blame the grand kids distracting me :)
). I posted my question on stackoverflow
(https://stackoverflow.com/questions/45579340/composite-persistence-unit-using-a-single-jar-file)
but so far I've only gotten a B.S. comment.

Basically, I need to know how to create a JPA composite persistence
unit composed of three MySQL databases that have relationships between
the three databases, without using three (or more) different JAR
files. It's for an e-commerce web site running under Glassfish
(probably Geronimo later) on my CentOS server.

More details of the problem on stackoverflow.

TIA,

PGA
--
Paul G. Allen, BSIT/SE
Owner, Sr. Engineer
Random Logic Consulting
www.randomlogic.com

--
KPLUG-***@kernel-panic.org
http://www.kernel-panic.org/cgi-bin/mailman/listinfo/kplug-list

Tony Su

2017-08-10 18:24:22 UTC

Permalink

First,
Storing data on more machines isn't necessarily going to make your
data more secure. Is your database being partitioned? Is each database
instance an exact copy of another? Method of data replication?

What you wrote gives hints but does not declare what you are setting up.

Only after you're clear what you are doing with your database, then
you can consider your options...

eg
j2ee connection pooling
Database connection middleware

IMO it's likely you may not be getting any stackflow responses because
you only know the name of the apps you're using and what you've
written doesn't display an understanding of architecture.

Saying this another way, before code can be written, you have to be
able to clearly describe your higher level requirements which include
design and objectives. In fact, that one response in its own way is
complaining about exactly this.

Trying to code(or looking to code) before you know what your code is
supposed to accomplish or if it's the best solution is an exercise in
useless head-banging.

Only after truly understanding(or describing) the whole picture you
<might> consider coding but there are also a lot of solutions that
don't really require any coding, only configuration of existing or
other off-the-shelf blocks of code (eg middleware).

And, in closing let me also mention the possibility that you may not
want any of this, your database application itself may support a lot
of what you want(ie data connections, query routing, data shipping and
other replication, etc). In fact, all modern database apps like MySQL,
PostgreSQL, DB2, etc can do this internally.

Tony

Post by Paul G. Allen
Anyone here an experienced J2EE engineer (I know there used to be a
few, but I've been semi-lurking for a while)?
I have an issue that I can't figure out. I'm told it can be done, I
found a link through Google with an example, and then I promptly lost
it (and can't find it again - I blame the grand kids distracting me :)
). I posted my question on stackoverflow
(https://stackoverflow.com/questions/45579340/composite-persistence-unit-using-a-single-jar-file)
but so far I've only gotten a B.S. comment.
Basically, I need to know how to create a JPA composite persistence
unit composed of three MySQL databases that have relationships between
the three databases, without using three (or more) different JAR
files. It's for an e-commerce web site running under Glassfish
(probably Geronimo later) on my CentOS server.
More details of the problem on stackoverflow.
TIA,
PGA
--
Paul G. Allen, BSIT/SE
Owner, Sr. Engineer
Random Logic Consulting
www.randomlogic.com
--
http://www.kernel-panic.org/cgi-bin/mailman/listinfo/kplug-list

--
KPLUG-***@kernel-panic.org
http://www.kernel-panic.org/cgi-bin/mailman/listinfo/kplug-list

Paul G. Allen

2017-08-10 20:06:51 UTC

Permalink

Perhaps you are correct.

PCI compliance requires that customer data be stored on a different
machine and be encrypted. Aside from that it's the smart thing to do
for security. Customer data is separated from the web server, as well
as product data, and split up, so that if someone gains access to the
system it's much more difficult to gain access to customer data (such
as addresses, passwords, credit card info, etc.) In short it IS
necessary to use multiple databases to separate customer (and user -
such as employees) data from store data (e.g. - inventory).

The coding was already done to support a single database, but that is
not secure enough (even with the customer data salted and encrypted).
Added more databases with connection pooling (which I already had with
the single DB) was simple as well - before I began cross-referencing
entities between the databases. Of course if I decided to NOT use the
JPA at all, and coded all the database handling myself (done that in
the past, before JPA existed), but that's a major PITA.

There is no connection middleware, only container managed DB
connection using the MySQL connector. (Type 4 JDBC driver)

To me it seems pretty simple (but maybe that's because of my inside
knowledge of what I'm doing? :) ): I want a composite persistence unit
(in persistence.xml) that includes three persistence units (PU) for
each of the data sources without having each persistence unit in a
separate JAR. One reason is to more easily avoid cyclic references
between the entities in the separate JARs (a common problem, as it
turns out). Another is to reduce the number of projects to be built
(one for each PU, one for the application with the composite PU, one
for a common set of super classes to avoid the cyclic references).
It's simple to have multiple persistence units as long as none of the
entities in each one reference the entities of any other, but as soon
as there are cross-references, things get more complicated. I've read
that it can be done, but I've seen nothing that really explains how.

FYI, OTS systems can't be used for this or I would have selected and
used one (would have made life easier, but then again most of them are
not secure enough anyway).

Thanks,

PGA

Post by Tony Su
First,
Storing data on more machines isn't necessarily going to make your
data more secure. Is your database being partitioned? Is each database
instance an exact copy of another? Method of data replication?
What you wrote gives hints but does not declare what you are setting up.
Only after you're clear what you are doing with your database, then
you can consider your options...
eg
j2ee connection pooling
Database connection middleware
IMO it's likely you may not be getting any stackflow responses because
you only know the name of the apps you're using and what you've
written doesn't display an understanding of architecture.
Saying this another way, before code can be written, you have to be
able to clearly describe your higher level requirements which include
design and objectives. In fact, that one response in its own way is
complaining about exactly this.
Trying to code(or looking to code) before you know what your code is
supposed to accomplish or if it's the best solution is an exercise in
useless head-banging.
Only after truly understanding(or describing) the whole picture you
<might> consider coding but there are also a lot of solutions that
don't really require any coding, only configuration of existing or
other off-the-shelf blocks of code (eg middleware).
And, in closing let me also mention the possibility that you may not
want any of this, your database application itself may support a lot
of what you want(ie data connections, query routing, data shipping and
other replication, etc). In fact, all modern database apps like MySQL,
PostgreSQL, DB2, etc can do this internally.
Tony

--
http://www.kernel-panic.org/cgi-bin/mailman/listinfo/kplug-list

--
Paul G. Allen, BSIT/SE
Owner, Sr. Engineer
Random Logic Consulting
www.randomlogic.com

--
KPLUG-***@kernel-panic.org
http://www.kernel-panic.org/cgi-bin/mailman/listinfo/kplug-list

Tony Su

2017-08-11 19:01:01 UTC

Permalink

To enable fault tolerance and possible improvements in performance,
I recommend configuring a MySQL cluster. Normally, you'd use an
ordinary database connection to the cluster which would have a common
cluster name.

As for partitioning your database...
That poses issues for directing your queries to the correct cluster or
single database. Depending on a number of possible parameters, this
can be straightforward or it might not. Only a Database
Administrator/Architect can design this properly.

As for encryption...
That's something that handled frontend, middleware or in the database
app. Although often seen, encrypting by the Application frontend is
usually bad because it potentially exposes your critical security to
the public Even if you don't use a specific middleware product, "best
practices" demands that this specific functionality be broken out of
the frontend app and likely into a defined middleware (even if
self-defined) to allow for future maintenance, possible modifications
and upgrades, not to also say you would be able to deploy on its own
network node/host. This function in particular is susceptible to flaws
if self-coded, the highest priority should be to use proven, off the
shelf code blocks (Let others go through the debugging process, not
you).

And more partitioning...
There are good and poor reasons for partitioning. Partitioning does
not really have anything to do with security, it's generally done to
improve performance. Encrypting whatever data as needed and storing
that and unencrypted data on same or separate machines is likely
little diff. More important is the method and type of encryption, and
how you identify the data. But, regardless of my opinion if you're
trying to comply with regulations, then you should investigate and
identify how compliance was previously achieved as the model you
should follow.

There are also a number of other techniques you can use to improve
security and thwart attacks like injection, most notably the use of
Stored Procedures (IIRC most recent versions of MySQL should support
SP for awhile now).. You can also network architect a solution to
restrict types of queries by inserting a Proxy server to filter
traffic content...

Tony

Post by Paul G. Allen
Perhaps you are correct.
PCI compliance requires that customer data be stored on a different
machine and be encrypted. Aside from that it's the smart thing to do
for security. Customer data is separated from the web server, as well
as product data, and split up, so that if someone gains access to the
system it's much more difficult to gain access to customer data (such
as addresses, passwords, credit card info, etc.) In short it IS
necessary to use multiple databases to separate customer (and user -
such as employees) data from store data (e.g. - inventory).
The coding was already done to support a single database, but that is
not secure enough (even with the customer data salted and encrypted).
Added more databases with connection pooling (which I already had with
the single DB) was simple as well - before I began cross-referencing
entities between the databases. Of course if I decided to NOT use the
JPA at all, and coded all the database handling myself (done that in
the past, before JPA existed), but that's a major PITA.
There is no connection middleware, only container managed DB
connection using the MySQL connector. (Type 4 JDBC driver)
To me it seems pretty simple (but maybe that's because of my inside
knowledge of what I'm doing? :) ): I want a composite persistence unit
(in persistence.xml) that includes three persistence units (PU) for
each of the data sources without having each persistence unit in a
separate JAR. One reason is to more easily avoid cyclic references
between the entities in the separate JARs (a common problem, as it
turns out). Another is to reduce the number of projects to be built
(one for each PU, one for the application with the composite PU, one
for a common set of super classes to avoid the cyclic references).
It's simple to have multiple persistence units as long as none of the
entities in each one reference the entities of any other, but as soon
as there are cross-references, things get more complicated. I've read
that it can be done, but I've seen nothing that really explains how.
FYI, OTS systems can't be used for this or I would have selected and
used one (would have made life easier, but then again most of them are
not secure enough anyway).
Thanks,
PGA

--
http://www.kernel-panic.org/cgi-bin/mailman/listinfo/kplug-list

--
Paul G. Allen, BSIT/SE
Owner, Sr. Engineer
Random Logic Consulting
www.randomlogic.com
--
http://www.kernel-panic.org/cgi-bin/mailman/listinfo/kplug-list

--
KPLUG-***@kernel-panic.org
http://www.kernel-panic.org/cgi-bin/mailman/listinfo/kplug-list

Kevin Keane Subscription

2017-08-12 06:32:40 UTC

Permalink

The database being on a different physical machine from the Web server (and with a firewall in between) makes a lot of sense, security-wise. Unless I’m missing something, Splitting the same data into three databases (which at all accessed essentially in the same way from the same Web server) doesn’t really seem to add much security. If a hacker manages to access one database despite the physical separation, then he can also access three the same way.

There is an exception: if the database with the sensitive information does not allow any queries, and all access is funneled through some kind of API (whether a REST front-end, or possibly views or stored procedures) then you *may* gain some security benefits – but such access control would likely prevent you from using standard database libraries; you’d be back to coding access to that database separately from the other two.

You also risk performance problems because instead of using joins at the database level, a lot of your database operations may have to be implemented at the application level, converting a single query potentially into dozens, and possibly also retrieving a lot of intermediate data. For instance, how do you retrieve a list of all items a customer ordered in the last year? You have to retrieve the customer order data from the customer database, and then loop over the inventory database to retrieve all the items he ordered.

That said, I’m not in your shoes, and don’t know all your requirements. It is very common that something that seems “wrong” on the surface makes a lot of sense in a particular business context. And partitioning the database like that may not cause many problems in your particular scenario.

Unfortunately, it has been 10 years since I did serious Java development, so I don’t know the answer to your question. But I suspect that the answer is “this isn’t a coding problem or JAR packaging problem, but an architecture problem”. In the end you would still have to code all the ways the three databases work together to deliver your data.

Sent from Mail for Windows 10

From: Paul G. Allen <mailto:***@gmail.com>
Sent: Thursday, August 10, 2017 1:08 PM
To: Main Discussion List for KPLUG <mailto:kplug-***@kernel-panic.org>
Subject: Re: Java programmers here?

Perhaps you are correct.

PCI compliance requires that customer data be stored on a different
machine and be encrypted. Aside from that it's the smart thing to do
for security. Customer data is separated from the web server, as well
as product data, and split up, so that if someone gains access to the
system it's much more difficult to gain access to customer data (such
as addresses, passwords, credit card info, etc.) In short it IS
necessary to use multiple databases to separate customer (and user -
such as employees) data from store data (e.g. - inventory).

The coding was already done to support a single database, but that is
not secure enough (even with the customer data salted and encrypted).
Added more databases with connection pooling (which I already had with
the single DB) was simple as well - before I began cross-referencing
entities between the databases. Of course if I decided to NOT use the
JPA at all, and coded all the database handling myself (done that in
the past, before JPA existed), but that's a major PITA.

There is no connection middleware, only container managed DB
connection using the MySQL connector. (Type 4 JDBC driver)

To me it seems pretty simple (but maybe that's because of my inside
knowledge of what I'm doing? :) ): I want a composite persistence unit
(in persistence.xml) that includes three persistence units (PU) for
each of the data sources without having each persistence unit in a
separate JAR. One reason is to more easily avoid cyclic references
between the entities in the separate JARs (a common problem, as it
turns out). Another is to reduce the number of projects to be built
(one for each PU, one for the application with the composite PU, one
for a common set of super classes to avoid the cyclic references).
It's simple to have multiple persistence units as long as none of the
entities in each one reference the entities of any other, but as soon
as there are cross-references, things get more complicated. I've read
that it can be done, but I've seen nothing that really explains how.

FYI, OTS systems can't be used for this or I would have selected and
used one (would have made life easier, but then again most of them are
not secure enough anyway).

Thanks,

PGA

--
http://www.kernel-panic.org/cgi-bin/mailman/listinfo/kplug-list

--
Paul G. Allen, BSIT/SE
Owner, Sr. Engineer
Random Logic Consulting
www.randomlogic.com

--
KPLUG-***@kernel-panic.org
http://www.kernel-panic.org/cgi-bin/mailman/listinfo/kplug-list

--
KPLUG-***@kernel-panic.org
http://www.kernel-panic.org/cgi-bin/mailman/listinfo/kplug-list