Easily find issues by searching: #<Issue ID>
Example: #1832
Easily find members by searching in: <username>, <first name> and <last name>.
Example: Search smith, will return results smith and adamsmith
| Dev |
|
| QA | swissarmyknife(*) |
| Priority | Minor |
| Complexity | Unknown |
| Component | Tools - Import Tool |
| Version | 16.0 |
For ParAccel, we tested the usage of Prepared Statements for bulk import to speed up the insertion process in v15. However, we discovered that the insertion time was the same w/ or w/o using Prepared Statements for ParAccel. ParAccel uses the PostgreSQL JDBC driver and recently a newer version has been released. We'll test w/ the newer version to see if we notice any performance improvements while using prepared statements. Interestingly enough, we did notice a significant performance improvement using Prepared Statements agains the PostgreSQL DB.
We actually use paraccel-jdbc for ParAccel. We downloaded the latest version PAJDBC-3.1.0-70004 from the Actian website. I tried testing batch import using prepared statements with this latest version and found no performance difference than using insert statements.
We know have access to Actian 5.0 and perform additional testing.
We know have access to Actian 5.0 and perform additional testing.
I tried testing batch import using prepared statements again and I found performance gain this time. I tried with ParAccel 4.0 and 5.0 and with both the old and new JDBC Driver and batch import using prepared statements is faster in all cases.
I enabled the usage of prepared statements for batch import for ParAccel and checked it in to v16.
QA, please verify with different data types.
I tried testing batch import using prepared statements again and I found performance gain this time. I tried with ParAccel 4.0 and 5.0 and with both the old and new JDBC Driver and batch import using prepared statements is faster in all cases.
I enabled the usage of prepared statements for batch import for ParAccel and checked it in to v16.
QA, please verify with different data types.
I tried performance testing for Batch and Full Transaction types for using import data in Paraccel (4.0 and 5.0) with in-build JDBC Driver on on ADS-15.0.11 and ADS-16.0.-dev-56. Please see results for Batch and Full Transaction type in below.
Transaction Type : Batch
| Elapsed Time /sec | ||||
| Databases | No. of Rows | Batch Size | ADS Version 15.0.11 | ADS Version 16.0.0-dev-56 |
|
Paraccel 4.0
|
5000 | 500 | 131 | 130 |
| 25000 | 500 | 641 | 701 | |
| 50000 | 500 | 1273 | 1312 | |
| 5000 | 1000 | 127 | 128 | |
| 25000 | 1000 | 633 | 645 | |
| 50000 | 1000 | 1267 | 1356 | |
| Elapsed Time /sec | ||||
| Databases | No. of Rows | Batch Size | ADS Version 15.0.11 | ADS Version 16.0.0-dev-56 |
|
Paraccel 5.0
|
5000 | 500 | 215 | 164 |
| 25000 | 500 | 842 | 834 | |
| 50000 | 500 | 2136 | 1656 | |
| 5000 | 1000 | 138 | 162 | |
| 25000 | 1000 | 813 | 810 | |
| 50000 | 1000 | 1830 | 1646 | |
Transaction Type : Full
| Elapsed Time /sec | ||||
| Databases | No. of Rows | Full import | ADS Version 15.0.11 | ADS Version 16.0.0-dev-56 |
| Paraccel 4.0 | 5000 | 330 | 324 | |
| 25000 | 1671 | 1655 | ||
| 50000 | More than 1 hrs | More than 1 hrs | ||
| Elapsed Time /sec | ||||
| Databases | No. of Rows | Full import | ADS Version 15.0.11 | ADS Version 16.0.0-dev-56 |
| Paraccel 5.0 | 5000 | 344 | 329 | |
| 25000 | 1684 | 1657 | ||
| 50000 | More than 1 hrs | More than 1 hrs | ||
I tried performance testing for Batch and Full Transaction types for using import data in Paraccel (4.0 and 5.0) with in-build JDBC Driver on on ADS-15.0.11 and ADS-16.0.-dev-56. Please see results for Batch and Full Transaction type in below.
Transaction Type : Batch
| Elapsed Time /sec | ||||
| Databases | No. of Rows | Batch Size | ADS Version 15.0.11 | ADS Version 16.0.0-dev-56 |
|
Paraccel 4.0
|
5000 | 500 | 131 | 130 |
| 25000 | 500 | 641 | 701 | |
| 50000 | 500 | 1273 | 1312 | |
| 5000 | 1000 | 127 | 128 | |
| 25000 | 1000 | 633 | 645 | |
| 50000 | 1000 | 1267 | 1356 | |
| Elapsed Time /sec | ||||
| Databases | No. of Rows | Batch Size | ADS Version 15.0.11 | ADS Version 16.0.0-dev-56 |
|
Paraccel 5.0
|
5000 | 500 | 215 | 164 |
| 25000 | 500 | 842 | 834 | |
| 50000 | 500 | 2136 | 1656 | |
| 5000 | 1000 | 138 | 162 | |
| 25000 | 1000 | 813 | 810 | |
| 50000 | 1000 | 1830 | 1646 | |
Transaction Type : Full
| Elapsed Time /sec | ||||
| Databases | No. of Rows | Full import | ADS Version 15.0.11 | ADS Version 16.0.0-dev-56 |
| Paraccel 4.0 | 5000 | 330 | 324 | |
| 25000 | 1671 | 1655 | ||
| 50000 | More than 1 hrs | More than 1 hrs | ||
| Elapsed Time /sec | ||||
| Databases | No. of Rows | Full import | ADS Version 15.0.11 | ADS Version 16.0.0-dev-56 |
| Paraccel 5.0 | 5000 | 344 | 329 | |
| 25000 | 1684 | 1657 | ||
| 50000 | More than 1 hrs | More than 1 hrs | ||
I've got ParAccel 5.1 as included in adstudio v16
Will there be a separate comparison for that version (ParAccel 5.1)?
I've got ParAccel 5.1 as included in adstudio v16
Will there be a separate comparison for that version (ParAccel 5.1)?
Changes in dev-67:
- Based on QA's performance measurements, we now use prepared statements for batch imports if it is ParAccel 5.0 and above.
- The latest ParAccel JDBC Driver (PAJDBC-3.1.0-70004) has been checked in.
Changes in dev-67:
- Based on QA's performance measurements, we now use prepared statements for batch imports if it is ParAccel 5.0 and above.
- The latest ParAccel JDBC Driver (PAJDBC-3.1.0-70004) has been checked in.
Issue #12509 |
| Closed |
| Fixed |
| Resolved |
Completion |
| No due date |
| Fixed Build ADS 16.0.0-dev-67 |
| No time estimate |
We actually use paraccel-jdbc for ParAccel. We downloaded the latest version PAJDBC-3.1.0-70004 from the Actian website. I tried testing batch import using prepared statements with this latest version and found no performance difference than using insert statements.