diff --git a/DESCRIPTION b/DESCRIPTION index e763421..d64937d 100644 --- a/DESCRIPTION +++ b/DESCRIPTION @@ -1,7 +1,7 @@ Package: bcputility Type: Package Title: Wrapper for SQL Server bcp Utility -Version: 0.4.3 +Version: 0.4.4 Authors@R: person("Thomas", "Roh", email = "thomas@roh.engineering", role = c("aut", "cre")) Description: Provides functions to utilize a command line utility that does bulk inserts and exports from SQL Server databases. License: MIT + file LICENSE diff --git a/NEWS.md b/NEWS.md index 71de574..2a3d0c0 100644 --- a/NEWS.md +++ b/NEWS.md @@ -1,3 +1,7 @@ +# bcputility 0.4.4 + +* Updated readme to clarify dependencies and provide quick start examples. + # bcputility 0.4.3 * An error is now thrown for cases where a projection has an EPSG that is `NA`. diff --git a/README.md b/README.md index 952062c..cf4baa3 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# bcputility +# bcputility [![CRAN status](https://www.r-pkg.org/badges/version/bcputility)](https://CRAN.R-project.org/package=bcputility) @@ -20,6 +20,11 @@ improves performance of large writes by using bulk inserts. An export function is provided for convenience, but likely will not significantly improve performance over other methods. +## Prerequisites + +The system dependencies can be downloaded and installed from +[Microsoft](https://learn.microsoft.com/en-us/sql/tools/bcp-utility#download-the-latest-version-of-the-bcp-utility). +It is recommended to add `bcp` and `sqlcmd` to the system path. ## Installation @@ -36,13 +41,38 @@ Install the development version with: devtools::install_github("tomroh/bcputility") ``` -If *bcp* and *sqlcmd* is not on the system path or you want to override the default, set the option with the full file path: +To check if the prerequisite binaries are on the path: + +```r +bcpVersion() +sqlcmdVersion() +``` + +If `bcp` and `sqlcmd` is not on the system path or you want to override the default, set the option with the full file path: ```r options(bcputility.bcp.path = "") options(bcputility.sqlcmd.path = "") ``` +## Usage + +Trusted Connection (default): + +```r +x <- read.csv("") +connectArgs <- makeConnectArgs(server = "", database = "") +bcpImport(x = x, connectargs = connectArgs, table = "") +``` + +SQL Authentication: + +```r +connectArgs <- makeConnectArgs(server = "", database = "", + username = "", password = "") +bcpImport(x = x, connectargs = connectArgs, table = table) +``` + ## Benchmarks Benchmarks were performed with a local installation of SQL Server Express. diff --git a/docs/404.html b/docs/404.html index 4090406..0e7ade7 100644 --- a/docs/404.html +++ b/docs/404.html @@ -34,7 +34,7 @@ bcputility - 0.4.3 + 0.4.4
specified default \"dbo\". fieldterminator character separator columns rowterminator character separator rows--new lines overwrite Whether overwrite table exists spatialtype spatial data type schema https://docs.microsoft.com/en-us/sql/relational-databases/spatial/spatial-data-types-overview, ignored x 'sf' object bcpOptions list additional options pass 'bcp' utility. See details. ... arguments pass system2","code":""},{"path":"https://bcputility.delveds.com/reference/bcpImport.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Import data to SQL Server — bcpImport","text":"Output system2. See ... redirect output.","code":""},{"path":"https://bcputility.delveds.com/reference/bcpImport.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Import data to SQL Server — bcpImport","text":"x dataframe object, data.table::fwrite used write memory object disk temporary file deleted function exits. fieldterminator rowterminator ignored case. overwrite TRUE, existing table name deleted schema inferred DBI::dbCreateTable. use customized schema, create schema calling function use overwrite=FALSE. x sf object, geometry column converted binary written database conversion geometry/geometry data type. EPSG code automatically read sf object used SRID. override default path bcp command line utility, set bcputility.bcp.path option. override default path sqlcmd command line utility, set bcputility.sqlcmd.path option. bcpOptions allows user include additional arguments call system2. Please refer https://learn.microsoft.com/en-us/sql/tools/bcp-utility. default options set defaults bcp CLI. -b refers number rows write time; 10,000 50,000 starting recommendation. -refers size packets sent bytes. -e refers maximum number errors failure.","code":""},{"path":"https://bcputility.delveds.com/reference/bcpExport.html","id":null,"dir":"Reference","previous_headings":"","what":"Export data from SQL Server — bcpExport","title":"Export data from SQL Server — bcpExport","text":"wrapper system call bcp utility writes SQL Server table query (T-SQL) file.","code":""},{"path":"https://bcputility.delveds.com/reference/bcpExport.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Export data from SQL Server — bcpExport","text":"","code":"bcpExport( file, connectargs, table, query, fieldterminator = \"\\t\", rowterminator = ifelse(.Platform$OS.type == \"windows\", \"\\r\\n\", \"\\n\"), bcpOptions = list(\"-c\", \"-b\", 1000, \"-a\", 4096, \"-m\", 10), ... )"},{"path":"https://bcputility.delveds.com/reference/bcpExport.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Export data from SQL Server — bcpExport","text":"file output file name connectargs named list connection arguments. See makeConnectArgs. table name source table exporting SQL Server query Transact-SQL query returns result set. Ignored table specified. fieldterminator character separator columns rowterminator character separator rows--new lines bcpOptions list additional options pass bcp utility. See details. ... arguments pass system2","code":""},{"path":"https://bcputility.delveds.com/reference/bcpExport.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Export data from SQL Server — bcpExport","text":"return value. Operations bcp printed console; see ... redirect output","code":""},{"path":"https://bcputility.delveds.com/reference/bcpExport.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Export data from SQL Server — bcpExport","text":"bcpOptions allows user include additional arguments call system2. Please refer https://learn.microsoft.com/en-us/sql/tools/bcp-utility. default options set defaults bcp CLI. -b refers number rows write time; 10,000 50,000 starting recommendation. -refers size packets sent bytes. -e refers maximum number errors failure.","code":""},{"path":"https://bcputility.delveds.com/reference/SQLServerCLIVersions.html","id":null,"dir":"Reference","previous_headings":"","what":"Check bcp and sqlcmd versions — SQLServerCLIVersions","title":"Check bcp and sqlcmd versions — SQLServerCLIVersions","text":"Check bcp sqlcmd versions","code":""},{"path":"https://bcputility.delveds.com/reference/SQLServerCLIVersions.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Check bcp and sqlcmd versions — SQLServerCLIVersions","text":"","code":"bcpVersion(...) sqlcmdVersion(...)"},{"path":"https://bcputility.delveds.com/reference/SQLServerCLIVersions.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Check bcp and sqlcmd versions — SQLServerCLIVersions","text":"... arguments pass system2","code":""},{"path":"https://bcputility.delveds.com/reference/createTable.html","id":null,"dir":"Reference","previous_headings":"","what":"Create or drop table — createTable","title":"Create or drop table — createTable","text":"Create drop table","code":""},{"path":"https://bcputility.delveds.com/reference/createTable.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Create or drop table — createTable","text":"","code":"createTable(connectargs, table, coltypes, ...) dropTable(connectargs, table, ...) checkTableExists(connectargs, table)"},{"path":"https://bcputility.delveds.com/reference/createTable.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Create or drop table — createTable","text":"connectargs named list connection arguments. See makeConnectArgs. table Name source table importing SQL Server. specifying schema table name see .
specified default \"dbo\". coltypes character vector data types column names list/vector names. Use mapDataTypes refer proper format. ... arguments pass system2","code":""},{"path":"https://bcputility.delveds.com/reference/createTable.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Create or drop table — createTable","text":"return value. Operations bcp printed console; see ... redirect output","code":""},{"path":"https://bcputility.delveds.com/reference/makeConnectArgs.html","id":null,"dir":"Reference","previous_headings":"","what":"Create a named list of connection arguments to translate to bcp and\r\nsqlcmd options — makeConnectArgs","title":"Create a named list of connection arguments to translate to bcp and\r\nsqlcmd options — makeConnectArgs","text":"Create named list connection arguments translate bcp sqlcmd options","code":""},{"path":"https://bcputility.delveds.com/reference/makeConnectArgs.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Create a named list of connection arguments to translate to bcp and\r\nsqlcmd options — makeConnectArgs","text":"","code":"makeConnectArgs( server, database, username, password, trustedconnection = TRUE, trustservercert = FALSE, azure = FALSE, quotedidentifiers = FALSE )"},{"path":"https://bcputility.delveds.com/reference/makeConnectArgs.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Create a named list of connection arguments to translate to bcp and\r\nsqlcmd options — makeConnectArgs","text":"server instance SQL Server connect database specifies database connect username login ID password password login ID trustedconnection use integrated security, username password required trustservercert trust server certificate azure use Azure Active Directory authentication, work integrated authentication. quotedidentifiers set QUOTED_IDENTIFIERS option '' connection bcp/sqlcmd SQL Server.","code":""},{"path":"https://bcputility.delveds.com/reference/makeConnectArgs.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Create a named list of connection arguments to translate to bcp and\r\nsqlcmd options — makeConnectArgs","text":"list connection arguments","code":""},{"path":"https://bcputility.delveds.com/reference/mapDataTypes.html","id":null,"dir":"Reference","previous_headings":"","what":"Determine SQL Server data types from data frame. Follows SQL Server\r\ndata type size constraints and chooses the smallest data type size. — mapDataTypes","title":"Determine SQL Server data types from data frame. Follows SQL Server\r\ndata type size constraints and chooses the smallest data type size. — mapDataTypes","text":"Determine SQL Server data types data frame. Follows SQL Server data type size constraints chooses smallest data type size.","code":""},{"path":"https://bcputility.delveds.com/reference/mapDataTypes.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Determine SQL Server data types from data frame. Follows SQL Server\r\ndata type size constraints and chooses the smallest data type size. — mapDataTypes","text":"","code":"mapDataTypes(x, coltypes) varChar(x) varBinary(x) int(x)"},{"path":"https://bcputility.delveds.com/reference/mapDataTypes.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Determine SQL Server data types from data frame. Follows SQL Server\r\ndata type size constraints and chooses the smallest data type size. — mapDataTypes","text":"x data.frame object coltypes vector names columns override default data type mapping","code":""},{"path":"https://bcputility.delveds.com/reference/mapDataTypes.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Determine SQL Server data types from data frame. Follows SQL Server\r\ndata type size constraints and chooses the smallest data type size. — mapDataTypes","text":"character vector names columns","code":""},{"path":"https://bcputility.delveds.com/reference/mapDataTypes.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Determine SQL Server data types from data frame. Follows SQL Server\r\ndata type size constraints and chooses the smallest data type size. — mapDataTypes","text":"","code":"mapDataTypes(data.frame( int = 1:5L, numeric = seq(0, 1, length.out = 5), character = LETTERS[1:5], factor = paste(LETTERS[1:5], LETTERS[1:5], sep = ''), logical = c(TRUE, FALSE, TRUE, FALSE, TRUE), date = seq(Sys.Date() - 4, Sys.Date(), 1L), datetime = seq(Sys.time() - 5, Sys.time(), length.out = 5) ) ) #> int numeric character factor logical date #> \"TINYINT\" \"FLOAT\" \"VARCHAR(1)\" \"VARCHAR(2)\" \"BIT\" \"DATE\" #> datetime #> \"DATETIME\""},{"path":"https://bcputility.delveds.com/authors.html","id":null,"dir":"","previous_headings":"","what":"Authors","title":"Authors and Citation","text":"Thomas Roh. Author, maintainer.","code":""},{"path":"https://bcputility.delveds.com/authors.html","id":"citation","dir":"","previous_headings":"","what":"Citation","title":"Authors and Citation","text":"Roh T (2024). bcputility: Wrapper SQL Server bcp Utility. https://bcputility.delveds.com.","code":"@Manual{bcputility, title = {bcputility: Wrapper for SQL Server bcp Utility}, author = {Thomas Roh}, year = {2024}, url = {https://bcputility.delveds.com}, }"},{"path":"https://bcputility.delveds.com/LICENSE.html","id":null,"dir":"","previous_headings":"","what":"MIT License","title":"MIT License","text":"Copyright (c) 2021 Thomas P. Roh Permission hereby granted, free charge, person obtaining copy software associated documentation files (“Software”), deal Software without restriction, including without limitation rights use, copy, modify, merge, publish, distribute, sublicense, /sell copies Software, permit persons Software furnished , subject following conditions: copyright notice permission notice shall included copies substantial portions Software. SOFTWARE PROVIDED “”, WITHOUT WARRANTY KIND, EXPRESS IMPLIED, INCLUDING LIMITED WARRANTIES MERCHANTABILITY, FITNESS PARTICULAR PURPOSE NONINFRINGEMENT. EVENT SHALL AUTHORS COPYRIGHT HOLDERS LIABLE CLAIM, DAMAGES LIABILITY, WHETHER ACTION CONTRACT, TORT OTHERWISE, ARISING , CONNECTION SOFTWARE USE DEALINGS SOFTWARE.","code":""},{"path":"https://bcputility.delveds.com/index.html","id":"bcputility-","dir":"","previous_headings":"","what":"Wrapper for SQL Server bcp Utility","title":"Wrapper for SQL Server bcp Utility","text":"bcputility wrapper command line utility program SQL Server bulk imports/exports. package assumes bcp already installed system search path. large inserts SQL Server ODBC connection (e.g. “DBI” package), writes can take long time row generates individual insert statement. bcp Utility greatly improves performance large writes using bulk inserts. export function provided convenience, likely significantly improve performance methods.","code":""},{"path":"https://bcputility.delveds.com/index.html","id":"installation","dir":"","previous_headings":"","what":"Installation","title":"Wrapper for SQL Server bcp Utility","text":"can install released version bcputility CRAN : Install development version : bcp sqlcmd system path want override default, set option full file path:","code":"install.packages(\"bcputility\") devtools::install_github(\"tomroh/bcputility\") options(bcputility.bcp.path = \"\") options(bcputility.sqlcmd.path = \"\")"},{"path":"https://bcputility.delveds.com/index.html","id":"benchmarks","dir":"","previous_headings":"","what":"Benchmarks","title":"Wrapper for SQL Server bcp Utility","text":"Benchmarks performed local installation SQL Server Express. testing remote SQL Server, performance bcp odbc improved.","code":""},{"path":"https://bcputility.delveds.com/index.html","id":"import","dir":"","previous_headings":"Benchmarks","what":"Import","title":"Wrapper for SQL Server bcp Utility","text":"Time seconds","code":"library(DBI) library(data.table) library(bcputility) server <- Sys.getenv('MSSQL_SERVER') database <- Sys.getenv('MSSQL_DB') driver <- 'ODBC Driver 17 for SQL Server' set.seed(11) n <- 1000000 importTable <- data.frame( int = sample(x = seq(1L, 10000L, 1L), size = n, replace = TRUE), numeric = sample(x = seq(0, 1, length.out = n/100), size = n, replace = TRUE), character = sample(x = state.abb, size = n, replace = TRUE), factor = sample(x = factor(x = month.abb, levels = month.abb), size = n, replace = TRUE), logical = sample(x = c(TRUE, FALSE), size = n, replace = TRUE), date = sample(x = seq(as.Date('2022-01-01'), as.Date('2022-12-31'), by = 'days'), size = n, replace = TRUE), datetime = sample(x = seq(as.POSIXct('2022-01-01 00:00:00'), as.POSIXct('2022-12-31 23:59:59'), by = 'min'), size = n, replace = TRUE) ) connectArgs <- makeConnectArgs(server = server, database = database) con <- DBI::dbConnect(odbc::odbc(), Driver = \"SQL Server\", Server = server, Database = database) importResults <- microbenchmark::microbenchmark( bcpImport1000 = { bcpImport(importTable, connectargs = connectArgs, table = 'importTable1', bcpOptions = list(\"-b\", 1000, \"-a\", 4096, \"-e\", 10), overwrite = TRUE, stdout = FALSE) }, bcpImport10000 = { bcpImport(importTable, connectargs = connectArgs, table = 'importTable2', bcpOptions = list(\"-b\", 10000, \"-a\", 4096, \"-e\", 10), overwrite = TRUE, stdout = FALSE) }, bcpImport50000 = { bcpImport(importTable, connectargs = connectArgs, table = 'importTable3', bcpOptions = list(\"-b\", 50000, \"-a\", 4096, \"-e\", 10), overwrite = TRUE, stdout = FALSE) }, bcpImport100000 = { bcpImport(importTable, connectargs = connectArgs, table = 'importTable4', bcpOptions = list(\"-b\", 100000, \"-a\", 4096, \"-e\", 10), overwrite = TRUE, stdout = FALSE) }, dbWriteTable = { con <- DBI::dbConnect(odbc::odbc(), Driver = driver, Server = server, Database = database, trusted_connection = 'yes') DBI::dbWriteTable(con, name = 'importTable5', importTable, overwrite = TRUE) }, times = 30L, unit = 'seconds' ) importResults"},{"path":"https://bcputility.delveds.com/index.html","id":"export-table","dir":"","previous_headings":"Benchmarks","what":"Export Table","title":"Wrapper for SQL Server bcp Utility","text":"Note: bcp exports data may match format fwrite. dateTimeAs = 'write.csv' used make timings comparable, decreased performance “data.table”. Optimized write formats date times fwrite outperforms bcp data small enough pulled memory. Time seconds","code":"exportResults <- microbenchmark::microbenchmark( bcpExportChar = { bcpExport('inst/benchmarks/test1.csv', connectargs = connectArgs, table = 'importTableInit', fieldterminator = ',', stdout = FALSE) }, bcpExportNchar = { bcpExport('inst/benchmarks/test2.csv', connectargs = connectArgs, table = 'importTableInit', fieldterminator = ',', stdout = FALSE) }, fwriteQuery = { fwrite(DBI::dbReadTable(con, 'importTableInit'), 'inst/benchmarks/test3.csv', dateTimeAs = 'write.csv', col.names = FALSE) }, times = 30L, unit = 'seconds' ) exportResults"},{"path":"https://bcputility.delveds.com/index.html","id":"export-query","dir":"","previous_headings":"Benchmarks","what":"Export Query","title":"Wrapper for SQL Server bcp Utility","text":"Time seconds","code":"query <- 'SELECT * FROM [dbo].[importTable1] WHERE int < 1000' queryResults <- microbenchmark::microbenchmark( bcpExportQueryChar = { bcpExport('inst/benchmarks/test4.csv', connectargs = connectArgs, query = query, fieldterminator = ',', stdout = FALSE) }, bcpExportQueryNchar = { bcpExport('inst/benchmarks/test5.csv', connectargs = connectArgs, query = query, fieldterminator = ',', stdout = FALSE) }, fwriteQuery = { fwrite(DBI::dbGetQuery(con, query), 'inst/benchmarks/test6.csv', dateTimeAs = 'write.csv', col.names = FALSE) }, times = 30L, unit = 'seconds' ) queryResults"},{"path":"https://bcputility.delveds.com/index.html","id":"import-geometry","dir":"","previous_headings":"Benchmarks","what":"Import Geometry","title":"Wrapper for SQL Server bcp Utility","text":"Importing spatial data ‘sf’ objects also supported. sql statements import produce equivalent tables database. Time seconds","code":"library(sf) nc <- st_read(system.file(\"gpkg/nc.gpkg\", package = \"sf\")) divN <- 10 shp1 <- cbind(nc[sample.int(nrow(nc), n / divN, replace = TRUE),], importTable[seq_len(n / divN), ], id = seq_len(n / divN)) geometryResults <- microbenchmark::microbenchmark( bcpImportGeometry = { bcpImport(shp1, connectargs = connectArgs, table = 'shp1', overwrite = TRUE, stdout = FALSE, spatialtype = 'geometry', bcpOptions = list(\"-b\", 50000, \"-a\", 4096, \"-m\", 0)) }, odbcImportGeometry = { con <- DBI::dbConnect(odbc::odbc(), driver = driver, server = server, database = database, trusted_connection = 'yes') tableName <- 'shp2' spatialType <- 'geometry' geometryColumn <- 'geom' binaryColumn <- 'geomWkb' srid <- sf::st_crs(nc)$epsg shpBin2 <- data.table(shp1) data.table::set(x = shpBin2, j = binaryColumn, value = blob::new_blob(lapply(sf::st_as_binary(shpBin2[[geometryColumn]]), as.raw))) data.table::set(x = shpBin2, j = geometryColumn, value = NULL) dataTypes <- DBI::dbDataType(con, shpBin2) dataTypes[binaryColumn] <- 'varbinary(max)' DBI::dbWriteTable(conn = con, name = tableName, value = shpBin2, overwrite = TRUE, field.types = dataTypes) DBI::dbExecute(conn = con, sprintf('alter table %1$s add %2$s %3$s;', tableName, geometryColumn, spatialType)) DBI::dbExecute(conn = con, sprintf('UPDATE %1$s SET geom = %3$s::STGeomFromWKB([%4$s], %2$d); ALTER TABLE %1$s DROP COLUMN [%4$s];', tableName, srid, spatialType, binaryColumn) ) }, bcpImportGeography = { bcpImport(shp1, connectargs = connectArgs, table = 'shp3', overwrite = TRUE, stdout = FALSE, spatialtype = 'geography', bcpOptions = list(\"-b\", 50000, \"-a\", 4096, \"-m\", 0)) }, odbcImportGeography = { con <- DBI::dbConnect(odbc::odbc(), driver = driver, server = server, database = database, trusted_connection = 'yes') tableName <- 'shp4' spatialType <- 'geography' geometryColumn <- 'geom' binaryColumn <- 'geomWkb' srid <- sf::st_crs(nc)$epsg shpBin4 <- data.table(shp1) data.table::set(x = shpBin4, j = binaryColumn, value = blob::new_blob(lapply(sf::st_as_binary(shpBin4[[geometryColumn]]), as.raw))) data.table::set(x = shpBin4, j = geometryColumn, value = NULL) dataTypes <- DBI::dbDataType(con, shpBin4) dataTypes[binaryColumn] <- 'varbinary(max)' DBI::dbWriteTable(conn = con, name = tableName, value = shpBin4, overwrite = TRUE, field.types = dataTypes) DBI::dbExecute(conn = con, sprintf('alter table %1$s add %2$s %3$s;', tableName, geometryColumn, spatialType)) DBI::dbExecute(conn = con, sprintf('UPDATE %1$s SET geom = %3$s::STGeomFromWKB([%4$s], %2$d); ALTER TABLE %1$s DROP COLUMN [%4$s];', tableName, srid, spatialType, binaryColumn) ) DBI::dbExecute(conn = con, sprintf( 'UPDATE %1$s SET [%2$s] = [%2$s].MakeValid().ReorientObject().MakeValid() WHERE [%2$s].MakeValid().EnvelopeAngle() > 90;', tableName, geometryColumn)) }, times = 30L, unit = 'seconds' ) geometryResults"},{"path":"https://bcputility.delveds.com/news/index.html","id":"bcputility-043","dir":"Changelog","previous_headings":"","what":"bcputility 0.4.3","title":"bcputility 0.4.3","text":"CRAN release: 2024-05-09 error now thrown cases projection EPSG NA. Added ability turn QUOTED_IDENTIFIERS. required Microsoft database products. Use bcp “-q” option. Column names quoted reserved words can used.","code":""},{"path":"https://bcputility.delveds.com/news/index.html","id":"bcputility-040","dir":"Changelog","previous_headings":"","what":"bcputility 0.4.0","title":"bcputility 0.4.0","text":"CRAN release: 2023-01-16 Override/set path sqlcmd options(bcputility.sqlcmd.path = \"\") Added bcpVersion sqlcmdVersion check versions. Specify tables without schema strictly character vector size 1 e.g. \".
\". schema table quoted although special characters recommended except “_“. Tests added can run local instance SQL Server. work without SQL Server client/server configuration. README updated new syntax updated benchmarks. bcpImport bcpExport gain bcpOptions argument allows user include options CLI. See documentation formatting specification. connections arguments bcpImport bcpExport deprecated–server, database, username, password, trustedconnection. Use makeConnectArgs connectargs argument adds Azure AD authentication option disregard trust server certificate. “DBI” dependency dropped SQL Server queries run using bcp sqlcmd. mapDataTypes convenience function determine SQL Server data type conversions. defaults attempt determine smallest type size column. createTable, dropTable, checkTableExists added internals convenience. primary purpose define empty table user-defined types bcpImport append . logicals longer need converted FALSE/TRUE 0/1 importing -memory data. argument azure added bcpImport bcpExport Azure Active Directory authentication. works username password authentication.","code":""},{"path":"https://bcputility.delveds.com/news/index.html","id":"bcputility-030","dir":"Changelog","previous_headings":"","what":"bcputility 0.3.0","title":"bcputility 0.3.0","text":"CRAN release: 2022-02-20 bcpImport now returns output system2 specifications stderr stdout added documentation using schema DBI::Id now supported trusted connections now compatible newer sql server drivers default drivers now trusted_connection='yes' user name passwords now quoted shQuote special characters handled across platforms added check see bcp program can found","code":""},{"path":"https://bcputility.delveds.com/news/index.html","id":"bcputility-020","dir":"Changelog","previous_headings":"","what":"bcputility 0.2.0","title":"bcputility 0.2.0","text":"CRAN release: 2021-08-06 set path bcp utility bcputility.bcp.path option added links source code, bug reports, documentation site rowterminator fieldterminator passed data.table::fwrite data memory Added support geometry/geography data import ‘sf’ objects","code":""},{"path":"https://bcputility.delveds.com/news/index.html","id":"bcputility-010","dir":"Changelog","previous_headings":"","what":"bcputility 0.1.0","title":"bcputility 0.1.0","text":"CRAN release: 2021-07-13 Added NEWS.md file track changes package. Added bcpImport import data SQL Server Added bcpExport export data SQL Server","code":""}] +[{"path":"https://bcputility.delveds.com/authors.html","id":null,"dir":"","previous_headings":"","what":"Authors","title":"Authors and Citation","text":"Thomas Roh. Author, maintainer.","code":""},{"path":"https://bcputility.delveds.com/authors.html","id":"citation","dir":"","previous_headings":"","what":"Citation","title":"Authors and Citation","text":"Roh T (2024). bcputility: Wrapper SQL Server bcp Utility. https://bcputility.delveds.com.","code":"@Manual{bcputility, title = {bcputility: Wrapper for SQL Server bcp Utility}, author = {Thomas Roh}, year = {2024}, url = {https://bcputility.delveds.com}, }"},{"path":"https://bcputility.delveds.com/LICENSE.html","id":null,"dir":"","previous_headings":"","what":"MIT License","title":"MIT License","text":"Copyright (c) 2021 Thomas P. Roh Permission hereby granted, free charge, person obtaining copy software associated documentation files (“Software”), deal Software without restriction, including without limitation rights use, copy, modify, merge, publish, distribute, sublicense, /sell copies Software, permit persons Software furnished , subject following conditions: copyright notice permission notice shall included copies substantial portions Software. SOFTWARE PROVIDED “”, WITHOUT WARRANTY KIND, EXPRESS IMPLIED, INCLUDING LIMITED WARRANTIES MERCHANTABILITY, FITNESS PARTICULAR PURPOSE NONINFRINGEMENT. EVENT SHALL AUTHORS COPYRIGHT HOLDERS LIABLE CLAIM, DAMAGES LIABILITY, WHETHER ACTION CONTRACT, TORT OTHERWISE, ARISING , CONNECTION SOFTWARE USE DEALINGS SOFTWARE.","code":""},{"path":"https://bcputility.delveds.com/index.html","id":"bcputility-","dir":"","previous_headings":"","what":"Wrapper for SQL Server bcp Utility","title":"Wrapper for SQL Server bcp Utility","text":"bcputility wrapper command line utility program SQL Server bulk imports/exports. package assumes bcp already installed system search path. large inserts SQL Server ODBC connection (e.g. “DBI” package), writes can take long time row generates individual insert statement. bcp Utility greatly improves performance large writes using bulk inserts. export function provided convenience, likely significantly improve performance methods.","code":""},{"path":"https://bcputility.delveds.com/index.html","id":"prerequisites","dir":"","previous_headings":"","what":"Prerequisites","title":"Wrapper for SQL Server bcp Utility","text":"system dependencies can downloaded installed Microsoft. recommended add bcp sqlcmd system path.","code":""},{"path":"https://bcputility.delveds.com/index.html","id":"installation","dir":"","previous_headings":"","what":"Installation","title":"Wrapper for SQL Server bcp Utility","text":"can install released version bcputility CRAN : Install development version : check prerequisite binaries path: bcp sqlcmd system path want override default, set option full file path:","code":"install.packages(\"bcputility\") devtools::install_github(\"tomroh/bcputility\") bcpVersion() sqlcmdVersion() options(bcputility.bcp.path = \"\") options(bcputility.sqlcmd.path = \"\")"},{"path":"https://bcputility.delveds.com/index.html","id":"usage","dir":"","previous_headings":"","what":"Usage","title":"Wrapper for SQL Server bcp Utility","text":"Trusted Connection (default): SQL Authentication:","code":"x <- read.csv(\"\") connectArgs <- makeConnectArgs(server = \"\", database = \"\") bcpImport(x = x, connectargs = connectArgs, table = \"
\") connectArgs <- makeConnectArgs(server = \"\", database = \"\", username = \"\", password = \"\") bcpImport(x = x, connectargs = connectArgs, table = table)"},{"path":"https://bcputility.delveds.com/index.html","id":"benchmarks","dir":"","previous_headings":"","what":"Benchmarks","title":"Wrapper for SQL Server bcp Utility","text":"Benchmarks performed local installation SQL Server Express. testing remote SQL Server, performance bcp odbc improved.","code":""},{"path":"https://bcputility.delveds.com/index.html","id":"import","dir":"","previous_headings":"Benchmarks","what":"Import","title":"Wrapper for SQL Server bcp Utility","text":"Time seconds","code":"library(DBI) library(data.table) library(bcputility) server <- Sys.getenv('MSSQL_SERVER') database <- Sys.getenv('MSSQL_DB') driver <- 'ODBC Driver 17 for SQL Server' set.seed(11) n <- 1000000 importTable <- data.frame( int = sample(x = seq(1L, 10000L, 1L), size = n, replace = TRUE), numeric = sample(x = seq(0, 1, length.out = n/100), size = n, replace = TRUE), character = sample(x = state.abb, size = n, replace = TRUE), factor = sample(x = factor(x = month.abb, levels = month.abb), size = n, replace = TRUE), logical = sample(x = c(TRUE, FALSE), size = n, replace = TRUE), date = sample(x = seq(as.Date('2022-01-01'), as.Date('2022-12-31'), by = 'days'), size = n, replace = TRUE), datetime = sample(x = seq(as.POSIXct('2022-01-01 00:00:00'), as.POSIXct('2022-12-31 23:59:59'), by = 'min'), size = n, replace = TRUE) ) connectArgs <- makeConnectArgs(server = server, database = database) con <- DBI::dbConnect(odbc::odbc(), Driver = \"SQL Server\", Server = server, Database = database) importResults <- microbenchmark::microbenchmark( bcpImport1000 = { bcpImport(importTable, connectargs = connectArgs, table = 'importTable1', bcpOptions = list(\"-b\", 1000, \"-a\", 4096, \"-e\", 10), overwrite = TRUE, stdout = FALSE) }, bcpImport10000 = { bcpImport(importTable, connectargs = connectArgs, table = 'importTable2', bcpOptions = list(\"-b\", 10000, \"-a\", 4096, \"-e\", 10), overwrite = TRUE, stdout = FALSE) }, bcpImport50000 = { bcpImport(importTable, connectargs = connectArgs, table = 'importTable3', bcpOptions = list(\"-b\", 50000, \"-a\", 4096, \"-e\", 10), overwrite = TRUE, stdout = FALSE) }, bcpImport100000 = { bcpImport(importTable, connectargs = connectArgs, table = 'importTable4', bcpOptions = list(\"-b\", 100000, \"-a\", 4096, \"-e\", 10), overwrite = TRUE, stdout = FALSE) }, dbWriteTable = { con <- DBI::dbConnect(odbc::odbc(), Driver = driver, Server = server, Database = database, trusted_connection = 'yes') DBI::dbWriteTable(con, name = 'importTable5', importTable, overwrite = TRUE) }, times = 30L, unit = 'seconds' ) importResults"},{"path":"https://bcputility.delveds.com/index.html","id":"export-table","dir":"","previous_headings":"Benchmarks","what":"Export Table","title":"Wrapper for SQL Server bcp Utility","text":"Note: bcp exports data may match format fwrite. dateTimeAs = 'write.csv' used make timings comparable, decreased performance “data.table”. Optimized write formats date times fwrite outperforms bcp data small enough pulled memory. Time seconds","code":"exportResults <- microbenchmark::microbenchmark( bcpExportChar = { bcpExport('inst/benchmarks/test1.csv', connectargs = connectArgs, table = 'importTableInit', fieldterminator = ',', stdout = FALSE) }, bcpExportNchar = { bcpExport('inst/benchmarks/test2.csv', connectargs = connectArgs, table = 'importTableInit', fieldterminator = ',', stdout = FALSE) }, fwriteQuery = { fwrite(DBI::dbReadTable(con, 'importTableInit'), 'inst/benchmarks/test3.csv', dateTimeAs = 'write.csv', col.names = FALSE) }, times = 30L, unit = 'seconds' ) exportResults"},{"path":"https://bcputility.delveds.com/index.html","id":"export-query","dir":"","previous_headings":"Benchmarks","what":"Export Query","title":"Wrapper for SQL Server bcp Utility","text":"Time seconds","code":"query <- 'SELECT * FROM [dbo].[importTable1] WHERE int < 1000' queryResults <- microbenchmark::microbenchmark( bcpExportQueryChar = { bcpExport('inst/benchmarks/test4.csv', connectargs = connectArgs, query = query, fieldterminator = ',', stdout = FALSE) }, bcpExportQueryNchar = { bcpExport('inst/benchmarks/test5.csv', connectargs = connectArgs, query = query, fieldterminator = ',', stdout = FALSE) }, fwriteQuery = { fwrite(DBI::dbGetQuery(con, query), 'inst/benchmarks/test6.csv', dateTimeAs = 'write.csv', col.names = FALSE) }, times = 30L, unit = 'seconds' ) queryResults"},{"path":"https://bcputility.delveds.com/index.html","id":"import-geometry","dir":"","previous_headings":"Benchmarks","what":"Import Geometry","title":"Wrapper for SQL Server bcp Utility","text":"Importing spatial data ‘sf’ objects also supported. sql statements import produce equivalent tables database. Time seconds","code":"library(sf) nc <- st_read(system.file(\"gpkg/nc.gpkg\", package = \"sf\")) divN <- 10 shp1 <- cbind(nc[sample.int(nrow(nc), n / divN, replace = TRUE),], importTable[seq_len(n / divN), ], id = seq_len(n / divN)) geometryResults <- microbenchmark::microbenchmark( bcpImportGeometry = { bcpImport(shp1, connectargs = connectArgs, table = 'shp1', overwrite = TRUE, stdout = FALSE, spatialtype = 'geometry', bcpOptions = list(\"-b\", 50000, \"-a\", 4096, \"-m\", 0)) }, odbcImportGeometry = { con <- DBI::dbConnect(odbc::odbc(), driver = driver, server = server, database = database, trusted_connection = 'yes') tableName <- 'shp2' spatialType <- 'geometry' geometryColumn <- 'geom' binaryColumn <- 'geomWkb' srid <- sf::st_crs(nc)$epsg shpBin2 <- data.table(shp1) data.table::set(x = shpBin2, j = binaryColumn, value = blob::new_blob(lapply(sf::st_as_binary(shpBin2[[geometryColumn]]), as.raw))) data.table::set(x = shpBin2, j = geometryColumn, value = NULL) dataTypes <- DBI::dbDataType(con, shpBin2) dataTypes[binaryColumn] <- 'varbinary(max)' DBI::dbWriteTable(conn = con, name = tableName, value = shpBin2, overwrite = TRUE, field.types = dataTypes) DBI::dbExecute(conn = con, sprintf('alter table %1$s add %2$s %3$s;', tableName, geometryColumn, spatialType)) DBI::dbExecute(conn = con, sprintf('UPDATE %1$s SET geom = %3$s::STGeomFromWKB([%4$s], %2$d); ALTER TABLE %1$s DROP COLUMN [%4$s];', tableName, srid, spatialType, binaryColumn) ) }, bcpImportGeography = { bcpImport(shp1, connectargs = connectArgs, table = 'shp3', overwrite = TRUE, stdout = FALSE, spatialtype = 'geography', bcpOptions = list(\"-b\", 50000, \"-a\", 4096, \"-m\", 0)) }, odbcImportGeography = { con <- DBI::dbConnect(odbc::odbc(), driver = driver, server = server, database = database, trusted_connection = 'yes') tableName <- 'shp4' spatialType <- 'geography' geometryColumn <- 'geom' binaryColumn <- 'geomWkb' srid <- sf::st_crs(nc)$epsg shpBin4 <- data.table(shp1) data.table::set(x = shpBin4, j = binaryColumn, value = blob::new_blob(lapply(sf::st_as_binary(shpBin4[[geometryColumn]]), as.raw))) data.table::set(x = shpBin4, j = geometryColumn, value = NULL) dataTypes <- DBI::dbDataType(con, shpBin4) dataTypes[binaryColumn] <- 'varbinary(max)' DBI::dbWriteTable(conn = con, name = tableName, value = shpBin4, overwrite = TRUE, field.types = dataTypes) DBI::dbExecute(conn = con, sprintf('alter table %1$s add %2$s %3$s;', tableName, geometryColumn, spatialType)) DBI::dbExecute(conn = con, sprintf('UPDATE %1$s SET geom = %3$s::STGeomFromWKB([%4$s], %2$d); ALTER TABLE %1$s DROP COLUMN [%4$s];', tableName, srid, spatialType, binaryColumn) ) DBI::dbExecute(conn = con, sprintf( 'UPDATE %1$s SET [%2$s] = [%2$s].MakeValid().ReorientObject().MakeValid() WHERE [%2$s].MakeValid().EnvelopeAngle() > 90;', tableName, geometryColumn)) }, times = 30L, unit = 'seconds' ) geometryResults"},{"path":"https://bcputility.delveds.com/reference/bcpExport.html","id":null,"dir":"Reference","previous_headings":"","what":"Export data from SQL Server — bcpExport","title":"Export data from SQL Server — bcpExport","text":"wrapper system call bcp utility writes SQL Server table query (T-SQL) file.","code":""},{"path":"https://bcputility.delveds.com/reference/bcpExport.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Export data from SQL Server — bcpExport","text":"","code":"bcpExport( file, connectargs, table, query, fieldterminator = \"\\t\", rowterminator = ifelse(.Platform$OS.type == \"windows\", \"\\r\\n\", \"\\n\"), bcpOptions = list(\"-c\", \"-b\", 1000, \"-a\", 4096, \"-m\", 10), ... )"},{"path":"https://bcputility.delveds.com/reference/bcpExport.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Export data from SQL Server — bcpExport","text":"file output file name connectargs named list connection arguments. See makeConnectArgs. table name source table exporting SQL Server query Transact-SQL query returns result set. Ignored table specified. fieldterminator character separator columns rowterminator character separator rows--new lines bcpOptions list additional options pass bcp utility. See details. ... arguments pass system2","code":""},{"path":"https://bcputility.delveds.com/reference/bcpExport.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Export data from SQL Server — bcpExport","text":"return value. Operations bcp printed console; see ... redirect output","code":""},{"path":"https://bcputility.delveds.com/reference/bcpExport.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Export data from SQL Server — bcpExport","text":"bcpOptions allows user include additional arguments call system2. Please refer https://learn.microsoft.com/en-us/sql/tools/bcp-utility. default options set defaults bcp CLI. -b refers number rows write time; 10,000 50,000 starting recommendation. -refers size packets sent bytes. -e refers maximum number errors failure.","code":""},{"path":"https://bcputility.delveds.com/reference/bcpImport.html","id":null,"dir":"Reference","previous_headings":"","what":"Import data to SQL Server — bcpImport","title":"Import data to SQL Server — bcpImport","text":"wrapper system call bcp utility bulk inserts SQL Server.","code":""},{"path":"https://bcputility.delveds.com/reference/bcpImport.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Import data to SQL Server — bcpImport","text":"","code":"bcpImport( x, connectargs, table, fieldterminator = \"\\t\", rowterminator = ifelse(.Platform$OS.type == \"windows\", \"\\r\\n\", \"\\n\"), overwrite = FALSE, spatialtype = c(\"geometry\", \"geography\"), bcpOptions = list(\"-b\", 1000, \"-a\", 4096, \"-m\", 10), ... )"},{"path":"https://bcputility.delveds.com/reference/bcpImport.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Import data to SQL Server — bcpImport","text":"x dataframe object path file connectargs named list connection arguments. See makeConnectArgs. table Name source table importing SQL Server. specifying schema table name see .
specified default \"dbo\". fieldterminator character separator columns rowterminator character separator rows--new lines overwrite Whether overwrite table exists spatialtype spatial data type schema https://docs.microsoft.com/en-us/sql/relational-databases/spatial/spatial-data-types-overview, ignored x 'sf' object bcpOptions list additional options pass 'bcp' utility. See details. ... arguments pass system2","code":""},{"path":"https://bcputility.delveds.com/reference/bcpImport.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Import data to SQL Server — bcpImport","text":"Output system2. See ... redirect output.","code":""},{"path":"https://bcputility.delveds.com/reference/bcpImport.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Import data to SQL Server — bcpImport","text":"x dataframe object, data.table::fwrite used write memory object disk temporary file deleted function exits. fieldterminator rowterminator ignored case. overwrite TRUE, existing table name deleted schema inferred DBI::dbCreateTable. use customized schema, create schema calling function use overwrite=FALSE. x sf object, geometry column converted binary written database conversion geometry/geometry data type. EPSG code automatically read sf object used SRID. override default path bcp command line utility, set bcputility.bcp.path option. override default path sqlcmd command line utility, set bcputility.sqlcmd.path option. bcpOptions allows user include additional arguments call system2. Please refer https://learn.microsoft.com/en-us/sql/tools/bcp-utility. default options set defaults bcp CLI. -b refers number rows write time; 10,000 50,000 starting recommendation. -refers size packets sent bytes. -e refers maximum number errors failure.","code":""},{"path":"https://bcputility.delveds.com/reference/SQLServerCLIVersions.html","id":null,"dir":"Reference","previous_headings":"","what":"Check bcp and sqlcmd versions — SQLServerCLIVersions","title":"Check bcp and sqlcmd versions — SQLServerCLIVersions","text":"Check bcp sqlcmd versions","code":""},{"path":"https://bcputility.delveds.com/reference/SQLServerCLIVersions.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Check bcp and sqlcmd versions — SQLServerCLIVersions","text":"","code":"bcpVersion(...) sqlcmdVersion(...)"},{"path":"https://bcputility.delveds.com/reference/SQLServerCLIVersions.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Check bcp and sqlcmd versions — SQLServerCLIVersions","text":"... arguments pass system2","code":""},{"path":"https://bcputility.delveds.com/reference/createTable.html","id":null,"dir":"Reference","previous_headings":"","what":"Create or drop table — createTable","title":"Create or drop table — createTable","text":"Create drop table","code":""},{"path":"https://bcputility.delveds.com/reference/createTable.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Create or drop table — createTable","text":"","code":"createTable(connectargs, table, coltypes, ...) dropTable(connectargs, table, ...) checkTableExists(connectargs, table)"},{"path":"https://bcputility.delveds.com/reference/createTable.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Create or drop table — createTable","text":"connectargs named list connection arguments. See makeConnectArgs. table Name source table importing SQL Server. specifying schema table name see .
specified default \"dbo\". coltypes character vector data types column names list/vector names. Use mapDataTypes refer proper format. ... arguments pass system2","code":""},{"path":"https://bcputility.delveds.com/reference/createTable.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Create or drop table — createTable","text":"return value. Operations bcp printed console; see ... redirect output","code":""},{"path":"https://bcputility.delveds.com/reference/makeConnectArgs.html","id":null,"dir":"Reference","previous_headings":"","what":"Create a named list of connection arguments to translate to bcp and\r\nsqlcmd options — makeConnectArgs","title":"Create a named list of connection arguments to translate to bcp and\r\nsqlcmd options — makeConnectArgs","text":"Create named list connection arguments translate bcp sqlcmd options","code":""},{"path":"https://bcputility.delveds.com/reference/makeConnectArgs.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Create a named list of connection arguments to translate to bcp and\r\nsqlcmd options — makeConnectArgs","text":"","code":"makeConnectArgs( server, database, username, password, trustedconnection = TRUE, trustservercert = FALSE, azure = FALSE, quotedidentifiers = FALSE )"},{"path":"https://bcputility.delveds.com/reference/makeConnectArgs.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Create a named list of connection arguments to translate to bcp and\r\nsqlcmd options — makeConnectArgs","text":"server instance SQL Server connect database specifies database connect username login ID password password login ID trustedconnection use integrated security, username password required trustservercert trust server certificate azure use Azure Active Directory authentication, work integrated authentication. quotedidentifiers set QUOTED_IDENTIFIERS option '' connection bcp/sqlcmd SQL Server.","code":""},{"path":"https://bcputility.delveds.com/reference/makeConnectArgs.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Create a named list of connection arguments to translate to bcp and\r\nsqlcmd options — makeConnectArgs","text":"list connection arguments","code":""},{"path":"https://bcputility.delveds.com/reference/mapDataTypes.html","id":null,"dir":"Reference","previous_headings":"","what":"Determine SQL Server data types from data frame. Follows SQL Server\r\ndata type size constraints and chooses the smallest data type size. — mapDataTypes","title":"Determine SQL Server data types from data frame. Follows SQL Server\r\ndata type size constraints and chooses the smallest data type size. — mapDataTypes","text":"Determine SQL Server data types data frame. Follows SQL Server data type size constraints chooses smallest data type size.","code":""},{"path":"https://bcputility.delveds.com/reference/mapDataTypes.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Determine SQL Server data types from data frame. Follows SQL Server\r\ndata type size constraints and chooses the smallest data type size. — mapDataTypes","text":"","code":"mapDataTypes(x, coltypes) varChar(x) varBinary(x) int(x)"},{"path":"https://bcputility.delveds.com/reference/mapDataTypes.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Determine SQL Server data types from data frame. Follows SQL Server\r\ndata type size constraints and chooses the smallest data type size. — mapDataTypes","text":"x data.frame object coltypes vector names columns override default data type mapping","code":""},{"path":"https://bcputility.delveds.com/reference/mapDataTypes.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Determine SQL Server data types from data frame. Follows SQL Server\r\ndata type size constraints and chooses the smallest data type size. — mapDataTypes","text":"character vector names columns","code":""},{"path":"https://bcputility.delveds.com/reference/mapDataTypes.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Determine SQL Server data types from data frame. Follows SQL Server\r\ndata type size constraints and chooses the smallest data type size. — mapDataTypes","text":"","code":"mapDataTypes(data.frame( int = 1:5L, numeric = seq(0, 1, length.out = 5), character = LETTERS[1:5], factor = paste(LETTERS[1:5], LETTERS[1:5], sep = ''), logical = c(TRUE, FALSE, TRUE, FALSE, TRUE), date = seq(Sys.Date() - 4, Sys.Date(), 1L), datetime = seq(Sys.time() - 5, Sys.time(), length.out = 5) ) ) #> int numeric character factor logical date #> \"TINYINT\" \"FLOAT\" \"VARCHAR(1)\" \"VARCHAR(2)\" \"BIT\" \"DATE\" #> datetime #> \"DATETIME\""},{"path":"https://bcputility.delveds.com/news/index.html","id":"bcputility-044","dir":"Changelog","previous_headings":"","what":"bcputility 0.4.4","title":"bcputility 0.4.4","text":"Updated readme clarify dependencies provide quick start examples.","code":""},{"path":"https://bcputility.delveds.com/news/index.html","id":"bcputility-043","dir":"Changelog","previous_headings":"","what":"bcputility 0.4.3","title":"bcputility 0.4.3","text":"CRAN release: 2024-05-09 error now thrown cases projection EPSG NA. Added ability turn QUOTED_IDENTIFIERS. required Microsoft database products. Use bcp “-q” option. Column names quoted reserved words can used.","code":""},{"path":"https://bcputility.delveds.com/news/index.html","id":"bcputility-040","dir":"Changelog","previous_headings":"","what":"bcputility 0.4.0","title":"bcputility 0.4.0","text":"CRAN release: 2023-01-16 Override/set path sqlcmd options(bcputility.sqlcmd.path = \"\") Added bcpVersion sqlcmdVersion check versions. Specify tables without schema strictly character vector size 1 e.g. \".
\". schema table quoted although special characters recommended except “_“. Tests added can run local instance SQL Server. work without SQL Server client/server configuration. README updated new syntax updated benchmarks. bcpImport bcpExport gain bcpOptions argument allows user include options CLI. See documentation formatting specification. connections arguments bcpImport bcpExport deprecated–server, database, username, password, trustedconnection. Use makeConnectArgs connectargs argument adds Azure AD authentication option disregard trust server certificate. “DBI” dependency dropped SQL Server queries run using bcp sqlcmd. mapDataTypes convenience function determine SQL Server data type conversions. defaults attempt determine smallest type size column. createTable, dropTable, checkTableExists added internals convenience. primary purpose define empty table user-defined types bcpImport append . logicals longer need converted FALSE/TRUE 0/1 importing -memory data. argument azure added bcpImport bcpExport Azure Active Directory authentication. works username password authentication.","code":""},{"path":"https://bcputility.delveds.com/news/index.html","id":"bcputility-030","dir":"Changelog","previous_headings":"","what":"bcputility 0.3.0","title":"bcputility 0.3.0","text":"CRAN release: 2022-02-20 bcpImport now returns output system2 specifications stderr stdout added documentation using schema DBI::Id now supported trusted connections now compatible newer sql server drivers default drivers now trusted_connection='yes' user name passwords now quoted shQuote special characters handled across platforms added check see bcp program can found","code":""},{"path":"https://bcputility.delveds.com/news/index.html","id":"bcputility-020","dir":"Changelog","previous_headings":"","what":"bcputility 0.2.0","title":"bcputility 0.2.0","text":"CRAN release: 2021-08-06 set path bcp utility bcputility.bcp.path option added links source code, bug reports, documentation site rowterminator fieldterminator passed data.table::fwrite data memory Added support geometry/geography data import ‘sf’ objects","code":""},{"path":"https://bcputility.delveds.com/news/index.html","id":"bcputility-010","dir":"Changelog","previous_headings":"","what":"bcputility 0.1.0","title":"bcputility 0.1.0","text":"CRAN release: 2021-07-13 Added NEWS.md file track changes package. Added bcpImport import data SQL Server Added bcpExport export data SQL Server","code":""}] diff --git a/docs/sitemap.xml b/docs/sitemap.xml index c58a692..423b901 100644 --- a/docs/sitemap.xml +++ b/docs/sitemap.xml @@ -1,42 +1,42 @@ - https://bcputility.delveds.com/reference/index.html + https://bcputility.delveds.com/authors.html - https://bcputility.delveds.com/reference/bcpImport.html + https://bcputility.delveds.com/LICENSE.html - https://bcputility.delveds.com/reference/bcpExport.html + https://bcputility.delveds.com/LICENSE-text.html - https://bcputility.delveds.com/reference/SQLServerCLIVersions.html + https://bcputility.delveds.com/index.html - https://bcputility.delveds.com/reference/createTable.html + https://bcputility.delveds.com/reference/index.html - https://bcputility.delveds.com/reference/makeConnectArgs.html + https://bcputility.delveds.com/reference/bcpExport.html - https://bcputility.delveds.com/reference/mapDataTypes.html + https://bcputility.delveds.com/reference/bcpImport.html - https://bcputility.delveds.com/news/index.html + https://bcputility.delveds.com/reference/SQLServerCLIVersions.html - https://bcputility.delveds.com/authors.html + https://bcputility.delveds.com/reference/createTable.html - https://bcputility.delveds.com/LICENSE-text.html + https://bcputility.delveds.com/reference/makeConnectArgs.html - https://bcputility.delveds.com/LICENSE.html + https://bcputility.delveds.com/reference/mapDataTypes.html - https://bcputility.delveds.com/index.html + https://bcputility.delveds.com/404.html - https://bcputility.delveds.com/404.html + https://bcputility.delveds.com/news/index.html