-3

what is DBF4 (dBase IV)(*.dbf) file fundamental format? And how can create these file in a same word editor as Notepad with typing?(Update:, or excel VBA?)

What is that's format specifications as:

  • Delimiter (Same as: , or tab or etc)
  • Separator (may Same as above!) (If these two are not synonymy)
  • Row End character: (Same as vbCrLf)
  • Defining headers of columns(fields).
  • Code-Page of encoding: (same as: Unicode - 1256 or etc)
  • and others...

Please present an algorithm for creating this DB file format that made us able to create a same file easily by a VBA method which creates a text file. (Update Or using built-in VBA or its references methods.)

I using below for creating text file.

Sub CsvExportRange(rngRange As Object, strFileName As String, strCharset, strSeparator As String, strRowEnd As String, NVC As Boolean) 'NVC: _
Null Value Control (If cell contain Null value, suppose reached end of range), d: delimiter

Dim rngRow As Range
Dim objStream As Object
Dim i, lngFR, lngLR As Long 'lngFR: First Row, lngLR: Last Row

lngFR = rngRange.SpecialCells(xlCellTypeVisible).Rows(1).row - rngRange.Rows(1).row + 1
lngLR = rngRange.End(xlDown).row - rngRange.Rows(1).row + 1

Set objStream = CreateObject("ADODB.Stream")
objStream.Type = 2
objStream.Charset = strCharset
objStream.Open

For i = lngFR To lngLR
    If Not (rngRange.Rows(i).EntireRow.Hidden) Then
        If IIf(NVC, (Cells(i + rngRange.Rows(1).row - 1, _
            rngRange.SpecialCells(xlCellTypeVisible).Columns(1).column).Value = vbNullString), False) Then Exit For
        objStream.WriteText CsvFormatRow(rngRange.Rows(i), strSeparator, strRowEnd)
    End If
Next i

objStream.SaveToFile strFileName, 2
objStream.Close
End Sub
Function CsvFormatRow(rngRow As Variant, strSeparator As String, strRowEnd As String) As String

Dim arrCsvRow() As String

ReDim arrCsvRow(rngRow.SpecialCells(xlCellTypeVisible).Cells.Count - 1)
Dim rngCell As Range
Dim lngIndex As Long

lngIndex = 0

For Each rngCell In rngRow.SpecialCells(xlCellTypeVisible).Cells
    arrCsvRow(lngIndex) = CsvFormatString(rngCell.Value, strSeparator)
    lngIndex = lngIndex + 1
Next rngCell

CsvFormatRow = Join(arrCsvRow, strSeparator) & strRowEnd

End Function
Function CsvFormatString(strRaw, strSeparator As String) As String

Dim boolNeedsDelimiting As Boolean

Dim strDelimiter, strDelimiterEscaped As String

strDelimiter = """"
strDelimiterEscaped = strDelimiter & strDelimiter

boolNeedsDelimiting = InStr(1, strRaw, strDelimiter) > 0 _
    Or InStr(1, strRaw, chr(10)) > 0 _
    Or InStr(1, strRaw, strSeparator) > 0

CsvFormatString = strRaw

If boolNeedsDelimiting Then
    CsvFormatString = strDelimiter & _
        Replace(strRaw, strDelimiter, strDelimiterEscaped) & _
        strDelimiter
End If

End Function

(Forgotten source)

Because I reached this: I should create a dbf file from my Excel Range by hand! After searching founded web sources.

Updated:

How can declare encoding of DBF?

About encoding that needed, considerable ones is Commonplace in this issue is Iran System encoding.

How can I store data with suitable encoding as Iran System in DB table records?

mgae2m
  • 1,097
  • 9
  • 37
  • Notepad can only save files as straight text. Why would you want to create a DBF file using Notepad?? And what has this got to do with VBA? – YowE3K Aug 31 '17 at 20:27
  • FWIW - does [this page](http://www.dbase.com/Knowledgebase/INT/db7_file_fmt.htm) help you with learning what the file format is? – YowE3K Aug 31 '17 at 20:34
  • 1
    have you researched your question on the internet? you did not .... i did a search and the first hit is a description of the format. – jsotola Aug 31 '17 at 20:34
  • 1
    I what create with Notepad because: I an in exporting my Excel `Range` to `DBF` stage, and got discouraged reaching a code for crating that exporter macro. So I'm trying found structure and algorithm of inside `DBF` for creating this file from my excel `Range` By hand with `VBA` macro. – mgae2m Aug 31 '17 at 20:36
  • you cannot use notepad ... the format is binary .... do a web search – jsotola Aug 31 '17 at 20:37
  • Notepad is not part of the Office suite - it does not have VBA available. – YowE3K Aug 31 '17 at 20:39
  • Correcting the question: Its mean due to the above code in question, that able creates `CSV` by customized `Delimiter` and `Row end` and `Charset`, I what to create a `DBF` file with controlled data entry in above sample macro. In fact referring to Notepad word was An example or simile for my purpose. – mgae2m Aug 31 '17 at 20:47
  • Why do you need to create a DBF file? Why not just create a text file (or some other format that dBASE accepts) and then load the data into dBASE? (And then have dBASE itself create the DBF file if you **have** to have one.) – YowE3K Aug 31 '17 at 20:47
  • I should create a report in DBF, and upload for each month. This report is created from a Excel `Range`. – mgae2m Aug 31 '17 at 20:49
  • A DBF file is in a binary format (as you can see if you follow the link I supplied earlier, or if you googled it yourself). It is not a text file in a comma-separated values format, or a tab-delimited format, or any other text format. – YowE3K Aug 31 '17 at 20:49
  • But why are you wanting to create the report as a database file? Where are you "uploading" the report to? – YowE3K Aug 31 '17 at 20:51
  • Thus I cant create this file with a regular algorithm and insert `Chars` in above method, algorithmic? – mgae2m Aug 31 '17 at 20:52
  • depends on what you mean by "regular algorithm" .... that said, you could do it with VBA, but, judging from your question, it is beyond your expertise. .... furthermore, i doubt that anyone here is willing to give the amount of time needed to develop something like that. – jsotola Aug 31 '17 at 20:55
  • Yes, you can use the VBA function `Chr` to write out a file containing the relevant codes and thus generate your own DBF exporter function. That is certainly do-able, and I wish you the best of luck if you choose to attempt it. – YowE3K Aug 31 '17 at 20:55
  • I upload this report for a government institute, and its duty of my company. We was creating this report with their Application. But because of multi reports should we prepare, I'm going to generate this report from our excel SpreadSheet. That institute receive our Insurance report in DBF format. – mgae2m Aug 31 '17 at 20:57
  • Why don't you upload the data into Access (you can do that from Excel), and then get Access to export as a DBF file (again, that can be controlled from Excel). It will be much, much, much, much, much easier than trying to write your own DBF file creator. – YowE3K Aug 31 '17 at 21:00
  • Or have you checked whether the application the government supplied you to use has an "import" function. (In my country, I have to submit forms to APRA, but instead of using their application to type in the numbers, I use the application's import function to read the data in, and then I just have to hit the "submit" button.) – YowE3K Aug 31 '17 at 21:04
  • @YowE3k@: I tried in [This](https://stackoverflow.com/questions/45968735/export-dbf-from-a-table-in-excel-with-consider-and-pay-attention-to-datatype-an) question, but not succeed.Thus i changed my approach. – mgae2m Aug 31 '17 at 21:09
  • Yes, I saw that answer, and I think it will do what you want. – YowE3K Aug 31 '17 at 21:13
  • @YowE3k@: Please write code to silent and quiet (not save file on the the disk) create an access object that involve my range in a table then create destination `DBF`. I cant found this way after try and search a day. Thus I think about developing a DBF creator with VBA. – mgae2m Aug 31 '17 at 21:16
  • @YowE3K@: Our government application, based on DBF4 (dBase IV)(*.dbf) and has internal Database. I cant import my Excel Range to that's database, So may they hadn't crate `import` function. I should create a macro in Excel VBA to export my reported Range in DBF. – mgae2m Aug 31 '17 at 21:28
  • @jsotola@: Confirming your letter, I was focus on "VBA+Excel+Range+DBF" keywords and not research on this matter. Because I did a look inside of DBF and all field headers and values was readable as a text file. So I though DBF format is simple text file as kind of Tab delimited or comma separated text file (same CSV). ;-( – mgae2m Aug 31 '17 at 21:39
  • If you want to create your own macro to generate a DBF file, I can't stop you. All I can do is wish you luck. (If I had a spare day or two, I might challenge myself to attempt it too - it's more complex than the usual things I do so it could be interesting and could involve a lot of research. But I don't have a spare day or two, so I'm not going to try.) – YowE3K Aug 31 '17 at 21:41
  • 1
    if you use Notepad++ then you can switch to _HEX_ display. that would show you the true structure of the file. .... or download this an use it to view the file contents https://sourceforge.net/projects/hexeditorvb/ – jsotola Aug 31 '17 at 21:43
  • Because what I did saw inside DBF file, I thought its an ANSI text file. All column headers and data values are readable certainly. Why this file is binary , but inside values are readable same as a Unicode 1256 or ANSI code page text file? – mgae2m Aug 31 '17 at 21:45
  • If you look at the file format specification, you will see that certain data types are stored as strings. (They will be "readable" in a text editor.) But other data types (such as "Double") are not stored as strings so they, and all the control information, are binary values. Your macro will need to generate all that control information correctly. (One slight mistake, and the whole file will be corrupt.) I have done this sort of thing before, and I estimate it will take me a couple of days. Before you start working on it, make an estimation of how long it will take you to finish. – YowE3K Aug 31 '17 at 21:48
  • The comments you are making in this question (e.g. not understanding that files can hold values that are "readable" as well as other "non-readable" characters) makes me guess that your estimate of the time to complete the macro should be of the order of several months. It is not a task for a novice. – YowE3K Aug 31 '17 at 21:52
  • I realized the issue and found DBF is not a text file. Hope finding answer for above right approach of this question. – mgae2m Aug 31 '17 at 21:59
  • 1
    there is this ... http://www.dbf2002.com/csv-converter/convert-csv-to-dbf.html – jsotola Aug 31 '17 at 22:59
  • @jsotola Ahh, but the OP doesn't want to create any temporary files in the process, otherwise the answer to their previous question would have solved the issue. (I'm not sure **why** that is a requirement - the operating system, and even Excel, is constantly creating temporary files. And the operating system often doesn't even delete them when it is finished with them!) – YowE3K Aug 31 '17 at 23:14
  • i did more checking .... this looks promising ..... https://stackoverflow.com/questions/322792/how-can-i-save-a-datatable-to-a-dbf – jsotola Aug 31 '17 at 23:14
  • @YowE3K, fyi. added code that produces a dbf file. – jsotola Sep 01 '17 at 02:15
  • @YowE3k@: My concern for emphasis on macro act done in memory is because process been quiet. Why `Access` object (`Set App = CreateObject("Access.Application")`) cannot involve DB and Tables in memory? So If this approach been quiet, I hadn't problem with Temp files. Last, @jsotola@'s method is exact and brief. – mgae2m Sep 01 '17 at 11:50

2 Answers2

2

we have joy .... lol

this test code creates a dbf file from data in excel worksheet

creates a table and inserts one record

Sub dbfTest()

' NOTE:  put this test data at top of worksheet (A1:F2)

' Name    Date        Code    Date2       Description    Amount
' frank  11/12/2017  234.00  11/20/2018   paint          $1.34



'   ref: microsoft activex data objects

    Dim path As String
    Dim fileName As String

    filePath = "C:\database\"
    fileName = "test"


    Dim dc As Range
    Dim typ As String
    Dim fieldName As String
    Dim createSql As String

    createSql = "create table " + fileName + " ("          ' the create table query produces the file in directory

    Dim a As Variant

    For Each dc In Range("a1:e1")

        fieldName = dc.Value
        a = dc.offset(1).Value

        Select Case VarType(a)
            Case vbString:   typ = "varchar(100)"
            Case vbBoolean:  typ = "varchar(10)"
            Case vbInteger:  typ = "int"
            Case vbLong:     typ = "Double"
            Case vbDate:     typ = "TimeStamp"
            Case Else:       typ = "varchar(5)"            ' default for undefined types
        End Select

        createSql = createSql + " [" + fieldName + "]" + " " + typ + ","

    Next dc

    createSql = Left(createSql, Len(createSql) - 1) + ")"

    Debug.Print createSql

    Dim conn As ADODB.connection
    Set conn = CreateObject("ADODB.Connection")

    conn.Open "DRIVER={Microsoft dBase Driver (*.dbf)};" & "DBQ=" & filePath                                    ' both work
'   conn.Open "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & filePath & ";Extended Properties=dBASE IV"

    Dim cmd As ADODB.Command
    Set cmd = CreateObject("ADODB.Command")

    cmd.ActiveConnection = conn

    cmd.CommandText = createSql
    cmd.Execute

    Dim insertSql As String
    insertSql = "insert into " + fileName + " values("

    For Each dc In Range("a2:e2")
        insertSql = insertSql + "'" + CStr(dc.Value) + "',"
    Next dc

    insertSql = Left(insertSql, Len(insertSql) - 1) + ")"

    Debug.Print insertSql

    cmd.CommandText = insertSql

    cmd.Execute

    conn.Close
    Set conn = Nothing

End Sub
jsotola
  • 2,141
  • 1
  • 6
  • 15
  • Your answer is brief, nice, clear and useful; and exact that's question needed. @jsotola@: I'm developing your code to ready for publish and will present you with it's uploaded path, here. Please leave it in [This question](https://stackoverflow.com/questions/45968735/export-dbf-from-a-table-in-excel-with-consider-and-pay-attention-to-datatype-an) as an answer. – mgae2m Sep 01 '17 at 11:56
  • @jsotola@: In above code, How can define `Code-Page` or `Encoding` as `Unicode`? – mgae2m Sep 01 '17 at 15:22
  • checking into it – jsotola Sep 01 '17 at 16:03
  • @jsotola@: I need record values (`CStr(dc.Value)`) with Unicode encoding. Which datatype can define Unicode to fields for this? I tried `typ = "nvarchar(100)"` for example, but `nvarchar` is not approved and return error. – mgae2m Sep 02 '17 at 04:03
  • @jsotola@: With your above routine, I have able to export Excel in DBF. But this not complete. I need define encoding in destination DBF. I reached coding table for suitable encoding, but how can define DBF encoding? – mgae2m Sep 03 '17 at 05:37
  • you mentioned in another post that you want _Iran System encoding_. i looked into it and it appears that it is not unicode. it is a way of encoding unicode characters into ascii. .... do you have an example dbf file that is considered to be _good_? maybe one with just one text field and only one record. – jsotola Sep 03 '17 at 06:16
  • by the way, there is a problem with storing actual unicode text into dbf, by using ADODB. there is no way to store a space character as unicode text into a field. .... the issue arises in the fact that unicode characters are composed of two bytes. .... in the space character, one of the bytes is a zero (0x00), which is an illegal character in a VBA string. – jsotola Sep 03 '17 at 06:22
  • Of course tomorrow, (about 8 hours later) I will share an example dbf file with _Iran System encoding_ That generated with government application. Please take a look at these questions: [1](https://stackoverflow.com/questions/46020751/made-dbf-with-specific-character-encoding-in-vba-procedure-with-declare-and-de) and [2](https://stackoverflow.com/questions/46022300/how-can-exchange-between-and-connect-cs-and-vb-for-application). I presented some ideas. – mgae2m Sep 03 '17 at 19:59
  • In my region, all experiences that I found in this matter, focused on _Iran System encoding_ and in CS -A Class that involving `Dictionary` generic list for characters mapping-. (As presented in [this](https://stackoverflow.com/questions/46022300/how-can-exchange-between-and-connect-cs-and-vb-for-application) question). Unfortunately not found any experiences in Unicode encoding, or VBA language and seems this may unique approach that I try program in VBA and Excel. – mgae2m Sep 03 '17 at 20:07
  • @jsotola VBA strings aren't null-terminated, so they can include a 0x00 character. (Whether a DBF file can handle it is another matter.) – YowE3K Sep 03 '17 at 21:59
  • @YowE3K it appears that ADODB does not like an `insert into ...` SQL statement that includes a null in the data string – jsotola Sep 04 '17 at 00:01
  • @jsotola That certainly sounds probable. Although VBA uses a string type that explicitly keeps track of the length of the string and therefore can have nulls within the string, most things don't like nulls because they have been built using null-terminated string types. – YowE3K Sep 04 '17 at 01:28
  • @jsotola [here](https://ufile.io/2b26f) I uploaded example.zip that contain 2 files: 1- `DSKWOR00.dbf` is generated with government program and 2-`example.xlsx` that I created involving 4 items has been compared default _windows_ encoding and _Iran System_ encoding that I give from `DSKWOR00.dbf`and my database in `example.xlsx` so each corresponding values (in a row) are equivalent, for evaluating them. – mgae2m Sep 04 '17 at 05:39
  • checking ... it is version 5 format .... ref. http://www.oocities.org/geoff_wass/dBASE/GaryWhite/dBASE/FAQ/qformt.htm .... the previous link that i posted is for version 7, so ignore that – jsotola Sep 04 '17 at 06:03
  • you can open the file with notepad++ ..... https://notepad-plus-plus.org/ .... it has a hex viewer ..... i do not know if you are able to download it – jsotola Sep 04 '17 at 06:05
  • I did installed Notepad++ without addition plug-ins (as Hex-Editor)and opened DSKWOR00.dbf ([Print Screen](https://ufile.io/9pntv)) Assuming that all ever right with `createSql` and `insertSql`, due to [this](https://stackoverflow.com/questions/46020751/made-dbf-with-specific-character-encoding-in-vba-procedure-with-declare-and-de) question and [dBASE III,IV,5 DBF file format](http://www.oocities.org/geoff_wass/dBASE/GaryWhite/dBASE/FAQ/qformt.htm) document, how can I set DBF character encoding(as _Iran System_ or _Unicode_, considering above approach for DBF generating.(`cmd.Execute` - `ADO`) – mgae2m Sep 04 '17 at 06:41
  • @jsotola How can I set DBF encoding as _iran System_ or _Unicode_ in use of this useful answer? – mgae2m Sep 04 '17 at 15:57
1

my research has concluded. the Iran System encoding is actually ascii, it is not unicode. it uses ascii values to represent some of the Persian alphabet.

the problem with converting from unicode to Iran System encoding is that any letter is written completely differently depending where in the word it is positioned. you have "isolated", "initial", "medial" and "final" forms of most of the letters.

it is like upper and lower case on steroids ... lol

ref: https://www.math.nmsu.edu/~mleisher/Software/csets/IRANSYSTEM.TXT

so additional process would be needed to convert unicode text in excel into an equivalent Iran System encoding string before storing in database.

the code creates a table with one text field and stores 3 records

Sub dbfTestWork()

'   ref: microsoft activex data objects

    Dim filePath As String
    Dim fileName As String

    filePath = "C:\database\"
    fileName = "test"

    Dim conn As ADODB.Connection
    Set conn = CreateObject("ADODB.Connection")

    conn.Open "Driver={Microsoft dBase Driver (*.dbf)};Dbq=" + filePath + ";"

    'conn.Open "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & filePath & ";Extended Properties=dBASE IV;"

    Dim fil As String
    fil = filePath & fileName & ".dbf"
    If Not Dir(fil, vbDirectory) = vbNullString Then Kill fil  ' delete file if it exists

    Dim cmd As ADODB.Command
    Set cmd = CreateObject("ADODB.Command")

    cmd.ActiveConnection = conn

    cmd.CommandText = "create table test ([testTextData] char(20))"
    cmd.Execute

    Dim nFileNum As Integer
    nFileNum = FreeFile                                                           ' Get an available file number from the system
    Open filePath & fileName & ".dbf" For Binary Lock Read Write As #nFileNum     ' Open the file in binary mode.  Locks are optional
    Put #nFileNum, 30, CByte(1)                                                   ' set language driver id (LDID)   0x01 = ascii encoding
    Close #nFileNum

'   Debug.Print Range("e2").Value

    Dim aaa As String
    aaa = StrConv(Range("e2").Value, vbUnicode)
'   Debug.Print aaa

    Dim cmdStr As String
    cmdStr = "insert into test values ('"

    Dim ccc As Variant
    For Each ccc In Array("ac", "92", "9e", "20", "93", "a1", "fe", "a4")   ' one of these two should store
        cmdStr = cmdStr & Chr(CDec("&h" & ccc))                             ' "good morning" in persian
    Next ccc
    cmdStr = cmdStr & "');"
    cmd.CommandText = cmdStr
    cmd.Execute

    cmdStr = "insert into test values ('"
    For Each ccc In Array("a4", "fe", "a1", "93", "20", "9e", "92", "ac")
        cmdStr = cmdStr & Chr(CDec("&h" & ccc))
    Next ccc
    cmdStr = cmdStr & "');"
    cmd.CommandText = cmdStr
    cmd.Execute

    cmd.CommandText = "insert into test values ('abc123');"
    cmd.Execute

    conn.Close
    Set conn = Nothing

End Sub
'
jsotola
  • 2,141
  • 1
  • 6
  • 15
  • Whats using `aaa = StrConv(Range("e2").Value, vbUnicode)`? – mgae2m Sep 05 '17 at 16:32
  • In above worthwhile answer, using [this](https://www.math.nmsu.edu/~mleisher/Software/csets/IRANSYSTEM.TXT) refrence with use of `CDec()`, I'm going to create `map` array for corresponding-equivalent _Unicode_ and _Iran System_ ASCII codes then converting this two text string with write appropriate function, use of above mapped dictionary and character recognize. then send you that result (when I succeed). – mgae2m Sep 05 '17 at 18:30
  • With great thanks, result of above discussion get ready to present, and I send prepared procedures, here to you. [PRESENT](https://ufile.io/fl9qz) Explain that In Iran country, there where a government monthly periodical task for sub constructors, that preparing insurance list report. I programmed the list generator for Iran governmental insurance list, with decoding their specified characters code, that same as Iran System encoding, with some differences in characters ASCW code. I programmed some of my last related question with your guidance for publish as an answer. with regard and thanks. – mgae2m Sep 21 '17 at 09:34
  • @, Please I owe you. – mgae2m Sep 22 '17 at 08:29