2016-08-04

setting up basic Django authentication

So one of the things that Django does not currently do (as of 1.10) is provide a built-in login screen template for your app (like web2py does), even though it provides the views and forms as part of django.contrib.auth, but that's most probably because of its use in the admin app. So here's how you'd setup a basic one, using the built-in views:

First, the default LOGIN_URL is the urlpath /accounts/login, which you can see in your Django's django/conf/global_settings.py. You can switch your project to use a different one. In django/contrib/auth/views.py, we notice that the login() defaults its template_name to 'registration/login.html'. You can switch this in your project's URL conf. In any case, if we stick to the defaults we need to add the following to our project's URL conf in myproject/urls.py:

from django.contrib.auth.views import login, logout

urlpatterns = [
    # the other urlpatterns for your project
    url(r'^accounts/login/$', login),
    url(r'^accounts/logout/$', logout),
]

Next, assuming that in myproject/settings.py, APP_DIRS is True, then we need to build in the page template for the form at myproject/myapp/templates/registration/login.html:

{% extends "registration/base.html" %}

{% block content %}

  <div class="container">

    {% if form.errors and not form.non_field_errors %}
      <p class="errornote">
      {% if form.errors.items|length == 1 %}
        "Please correct the error below."
      {% else %}
        "Please correct the errors below."
      {% endif %}
      </p>
    {% endif %}

    {% if form.non_field_errors %}
      {% for error in form.non_field_errors %}
        <p class="errornote">
        {{ error }}
        </p>
      {% endfor %}
    {% endif %}

    <form class="form-signin" method="POST">
      {% csrf_token %}
      <h2 class="form-signin-heading">Please sign in</h2>
      <label for="inputUser" class="sr-only">User Name</label>
      <input type="text" name="username" id="inputUser" class="form-control" placeholder="User Name" maxlength="151" required autofocus>
      <label for="inputPassword" class="sr-only">Password</label>
      <input type="password" name="password" id="inputPassword" class="form-control" placeholder="Password" required>
      <button class="btn btn-lg btn-primary btn-block" type="submit">Sign in</button>
    </form>
  
  </div>

{% endblock %}

NOTE: Because the builtin-view login is called, which renders the builtin AuthenticationForm, you don't specify a form action in your login form template. Doing so will result in a Error NoReverseMatch on Reverse Django URL!

Then in the view that you want to protect:

from django.http import HttpResponse
from django.contrib.auth.decorators import login_required

@login_required
def index(request):
    # yadda
    return HttpResponse(yourviewtemplate.render(Context()))

Then when you go to the index view, you get a screen that looks like:

Note that the base.html references bootstrap.css and friends

auto register all Django models in admin

In Django admin if you want to autoload all app models in admin without having to enumerate each one:

Instead of what the tutorial writes:

from django.contrib import admin
from .models import Publisher, Author, Book

admin.site.register(Publisher)
admin.site.register(Author)
admin.site.register(Book)

instead:

from django.contrib import admin
from . import models
import inspect

for name, obj in inspect.getmembers(models):
    if inspect.isclass(obj):
        admin.site.register(getattr(models, name))

2016-08-01

FreeBSD virtualenv (for things like django)

Useful: http://kdebowski.pl/blog/freebsd-nginx-gunicorn-django-virtualenv-mysql/

System Era: FreeBSD 10.3
PYTHON_VERSION=python3.4

Remember to install databases/py-sqlite3 !

KEY THING after you've installed devel/py-virtualenv:

% virtualenv-3.4 --no-site-packages path_to_virtualenv/virtualenv_name
% source path_to_virtualenv/virtualenv_name/bin/activate.csh

Now you're in the virtualenv and you can pip and so forth.

2015-10-14

VZW Droid4 tethering enable

Enable tethering on a VZW Droid4:

use aSQLiteManager/aShell to edit:

/data/data/com.motorola.android.providers.settings/settings.db

set:

entitlement.check = 0

2015-10-13

where is the backup su stored by Voodoo OTA RootKeeper?

When you make a backup of su in Voodoo OTA RootKeeper 2.0.3 on Android 4.1.2 where does it go?
/system/usr/we-need-root/su-backup

I took a statically-linked version of an su which literally just changes the uid to 0 when its run setuid and stuck it there too, named unrestricted-su (it doesn't linkup with SuperSU for privilege prompts or anything).

Note this only works on Android < 4.2 since 4.3 mounts /system setuid and does not allow dalvik zygotes to run setuid binaries.

Forcing windows 8.1 to install google usb drivers for adb access

http://stackoverflow.com/a/25576781/2718295
I had the following problem: I had a Android phone without drivers, and it could not be recognized by the Windows 8.1. Neither as phone, neither as USB storage device. I searched Device manager. I opened Device manager, I right click on Android Phone->Android Composite Interface. I selected "Update Driver Software" I choose "Browse My Computer for Driver Software" Then I choose "Let me pick from a list of devices" I selected "USB Composite Device" A new USB device is added to the list, and I can connect to my phone using adb and Android SDK. Also I can use the phone as storage device. Good luck

2015-08-27

Get the DDL for a table in MS SQL Server

Query 1 gets you the actual column lengths from http://stackoverflow.com/questions/3854730/how-to-get-the-length-of-char-or-varchar-field-in-sql-server
SELECT
    sh.name+'.'+o.name AS ObjectName
        ,o.type_desc AS ObjectType
        ,s.name as ColumnName
        ,CASE
             WHEN t.name IN ('char','varchar') THEN t.name+'('+CASE WHEN s.max_length<0 then 'MAX' ELSE CONVERT(varchar(10),s.max_length) END+')'
             WHEN t.name IN ('nvarchar','nchar') THEN t.name+'('+CASE WHEN s.max_length<0 then 'MAX' ELSE CONVERT(varchar(10),s.max_length/2) END+')'
            WHEN t.name IN ('numeric') THEN t.name+'('+CONVERT(varchar(10),s.precision)+','+CONVERT(varchar(10),s.scale)+')'
             ELSE t.name
         END AS DataType
        ,CASE
             WHEN s.is_nullable=1 THEN 'NULL'
            ELSE 'NOT NULL'
        END AS Nullable
        ,CASE
             WHEN ic.column_id IS NULL THEN ''
             ELSE ' identity('+ISNULL(CONVERT(varchar(10),ic.seed_value),'')+','+ISNULL(CONVERT(varchar(10),ic.increment_value),'')+')='+ISNULL(CONVERT(varchar(10),ic.last_value),'null')
         END
        +CASE
             WHEN sc.column_id IS NULL THEN ''
             ELSE ' computed('+ISNULL(sc.definition,'')+')'
         END
        +CASE
             WHEN cc.object_id IS NULL THEN ''
             ELSE ' check('+ISNULL(cc.definition,'')+')'
         END
            AS MiscInfo
    FROM sys.columns                           s
        INNER JOIN sys.types                   t ON s.system_type_id=t.system_type_id and t.is_user_defined=0
        INNER JOIN sys.objects                 o ON s.object_id=o.object_id
        INNER JOIN sys.schemas                sh on o.schema_id=sh.schema_id
        LEFT OUTER JOIN sys.identity_columns  ic ON s.object_id=ic.object_id AND s.column_id=ic.column_id
        LEFT OUTER JOIN sys.computed_columns  sc ON s.object_id=sc.object_id AND s.column_id=sc.column_id
        LEFT OUTER JOIN sys.check_constraints cc ON s.object_id=cc.parent_object_id AND s.column_id=cc.parent_column_id
    WHERE o.name='YourTableName'
    order by 1,s.column_id

Query 2 gets you column names too but preserves the column order of the referenced table: http://stackoverflow.com/questions/1054984/how-can-i-get-column-names-from-a-table-in-sql-server
SELECT o.Name                   as Table_Name
     , c.Name                   as Field_Name
     , t.Name                   as Data_Type
     , t.length                 as Length_Size
     , t.prec                   as Precision_
FROM syscolumns c 
     INNER JOIN sysobjects o ON o.id = c.id
     LEFT JOIN  systypes t on t.xtype = c.xtype  
WHERE o.type = 'U' 
and o.Name = 'ctpt_upsebill_load'
--ORDER BY o.Name, c.Name

Use the above 2 queries as CTLs and join Query 1 to Query 2 to get full DDL in the correct order.

2015-05-04

Read file pattern (Python)

Here's a pattern that I've been using a lot to read files in Python 2.7:
import codecs
badrow_count = 0
goodrow_count = 0
rowcount = 0

filename = "somefile.txt"
encoding = "utf-8"
# see https://docs.python.org/2/library/codecs.html#standard-encodings
# for encodings

decode_error_handler = 'strict'
# this is the default
# see https://docs.python.org/2/library/codecs.html#codec-base-classes 
# for decoding error callbacks

f = codecs.open(filename=filename, mode='rU', encoding=encoding, 
    errors=decode_error_handler)
eof = False
while not eof:
    row = u''
    try:
        row = f.next()
    except UnicodeDecodeError as e:
        badrow_count += 1
        # do other things on this row
    except StopIteration:
        eof = True
    #except Exception as e:
        # handle other issues    
    else:
        goodrow_count += 1
        # do other stuff with row
    finally:
        if not eof:
            rowcounter += 1
        else:
            break
I prefer this to:
for row in f:
primarily in order to catch unicode decoder errors.

2015-02-27

postgresql unpivot

Given a monthly currency conversion table like:
iso4127_codeJanFebMarAprMayJunJulAugSepOctNovDec
JPY0.0096130.0097920.0097780.0097570.0098270.0097900.0098290.0097170.0093460.0092620.0086060.008375

I need to unpivot so that I get a table that is:
iso4127_codemonthrate
JPY10.009613
JPY20.009792
...
select iso4127_code, 
unnest(array[1,2,3,4,5,6,7,8,9,10,11,12]) as month,
unnest(array[jan, feb, mar, apr, may, jun, jul, aug, sep, oct, nov, dec]) as rate
from table

Strip HTML tags using regex

While for proper Markup Language parsing one should actually use an XML parser, sometimes you just want to strip all the markup using regex:
s/<\/?[^>]+>//

Query AD using ldapsearch

(Useful for troubleshooting AD logons if you have cygwin:

http://jurjenbokma.com/ApprenticesNotes/ldapsearch_ad_query.html)

Now at: http://jurjenbokma.com/ApprenticesNotes/ldapsearch_ad_query.xhtml
to force TLS_REQCERT never:

LDAPTLS_REQCERT=never ldapsearch ...

2014-12-31

Dynamic/Metaprogramming in ABAP

http://scn.sap.com/thread/1374403#7695723 http://scn.sap.com/thread/699312

2014-10-28

Preferred SSLCipherSuite for mod_ssl

SSLCipherSuite DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:HIGH:!3DES:!ECDH:!SRP:!aNULL:!CAMELLIA:!PSK:!EXPORT:!eNULL

Provides:
DHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(128) Mac=AEAD
DHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(256) Mac=AEAD
DHE-RSA-AES128-SHA256   TLSv1.2 Kx=DH       Au=RSA  Enc=AES(128)  Mac=SHA256
DHE-RSA-AES256-SHA256   TLSv1.2 Kx=DH       Au=RSA  Enc=AES(256)  Mac=SHA256
DHE-RSA-AES128-SHA      SSLv3 Kx=DH       Au=RSA  Enc=AES(128)  Mac=SHA1
DHE-RSA-AES256-SHA      SSLv3 Kx=DH       Au=RSA  Enc=AES(256)  Mac=SHA1
AES128-GCM-SHA256       TLSv1.2 Kx=RSA      Au=RSA  Enc=AESGCM(128) Mac=AEAD
AES256-GCM-SHA384       TLSv1.2 Kx=RSA      Au=RSA  Enc=AESGCM(256) Mac=AEAD
AES128-SHA256           TLSv1.2 Kx=RSA      Au=RSA  Enc=AES(128)  Mac=SHA256
AES256-SHA256           TLSv1.2 Kx=RSA      Au=RSA  Enc=AES(256)  Mac=SHA256
AES128-SHA              SSLv3 Kx=RSA      Au=RSA  Enc=AES(128)  Mac=SHA1
AES256-SHA              SSLv3 Kx=RSA      Au=RSA  Enc=AES(256)  Mac=SHA1
DHE-DSS-AES256-GCM-SHA384 TLSv1.2 Kx=DH       Au=DSS  Enc=AESGCM(256) Mac=AEAD
DHE-DSS-AES256-SHA256   TLSv1.2 Kx=DH       Au=DSS  Enc=AES(256)  Mac=SHA256
DHE-DSS-AES256-SHA      SSLv3 Kx=DH       Au=DSS  Enc=AES(256)  Mac=SHA1
DHE-DSS-AES128-GCM-SHA256 TLSv1.2 Kx=DH       Au=DSS  Enc=AESGCM(128) Mac=AEAD
DHE-DSS-AES128-SHA256   TLSv1.2 Kx=DH       Au=DSS  Enc=AES(128)  Mac=SHA256
DHE-DSS-AES128-SHA      SSLv3 Kx=DH       Au=DSS  Enc=AES(128)  Mac=SHA1
This is a modification of the ciphersuite list from http://www.matthewgkeller.com/blog/2014/01/09/ecdhe-vs-dhe-in-the-new-world-order/comment-page-1/

Primarily we are looking to remove elliptical curve ciphers in favor of discrete log methods (Schneier, 2013) due to the uncertainty of NSA compromization of ECC. We also try to prioritize the remaining available ciphers by preferring GCM mode over CBC mode.

  1. AES128 has better key schedule than AES256 (Schneier, 2013).
  2. We support "SSLv3" ciphers because in OpenSSL, TLSv1.0 ciphers are classified as SSLv3 ciphers.

2014-10-27

SAP Internal Order Settlement Receiver Field

The value of this field depends on a business logic ruleset (thus, SAP GUI will report it as a structured field) when you inspect it from Internal Order display/Settlement rule tab. Here is (part of) the ruleset for determining the Settlement Receiver value: (Pseudo-SQL, NOT ABAP)
CASE 
   WHEN COBRB.KONTY = 'CTR'
      THEN COBRB.KOSTL
   WHEN COBRB.KONTY = 'FXA'
      THEN COBRB.ANLN1
END
(FXA is fixed asset settlement).

h/t (http://scn.sap.com/thread/3416443)

2014-10-22

Turn off the login menu in web2py

Use Case: when you do not need to manage users or access control in the web2py application, remove the login menu from the right side of the menubar: Remove instantiating Auth(db) in your model and remove all dependencies on the Auth object.

web2py admin behind Apache proxy

So I have this web2py configuration where Apache httpd SSL proxies URLs of the form https://server/web2py/ to http://localhost:8081/web2py (web2py Rocket):
RewriteEngine on

RewriteCond %{HTTPS} !=on
RewriteCond %{HTTP_HOST} !=localhost
RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]

RewriteRule ^/webpy$ /web2py/ [R,L]

ProxyRequests off
ProxyPass /web2py/ http://localhost:8081/web2py/

<Location /web2py/>
    ProxyPassReverse http://localhost:8081/web2py/
</Location>
This includes a modified global routes.py:
default_application = 'init'    # ordinarily set in base routes.py
default_controller = 'default'  # ordinarily set in app-specific routes.py
default_function = 'index'      # ordinarily set in app-specific routes.py

BASE = '/web2py'

routes_in = (
    # do not reroute admin unless you want to disable it
    (BASE + '/admin', '/admin/default/index'),
    (BASE + '/admin/$anything', '/admin/$anything'),
    # do not reroute appadmin unless you want to disable it
    (BASE + '/$app/appadmin', '/$app/appadmin/index'),
    (BASE + '/$app/appadmin/$anything', '/$app/appadmin/$anything'),
    # do not reroute static files
    (BASE + '/$app/static/$anything', '/$app/static/$anything'),
    # reroute favicon and robots, use exable for lack of better choice
    ('/favicon.ico', '/examples/static/favicon.ico'),
    ('/robots.txt', '/examples/static/robots.txt'),
    # do other stuff
    ((r'.*http://otherdomain\.com.* (?P.*)', r'/app/ctr\g')),
    # remove the BASE prefix
    (BASE + '/$anything', '/$anything'),
)

routes_out = [(x, y) for (y, x) in routes_in]

logging = 'debug'

#fix ticket routing
error_message = '<html><body><h1>%s</h1></body></html>'
error_message_ticket = '<html><body><h1>Internal error</h1>Ticket issued: <a href="' + BASE + '/admin/default/ticket/%(ticket)s" target="_blank">%(ticket)s</a></body></html>'

def __routes_doctest():
    pass

if __name__ == '__main__':
    import doctest
    doctest.testmod()
But if you go to https://hostname/web2py/admin, it returns Admin is disabled because insecure channel. However, the channel IS secure since we are using SSL via the Apache. Offending Lines of code: applications/admin/models/access.py:
if request.is_https:
    session.secure()
elif not request.is_local and not DEMO_MODE:
    raise HTTP(200, T('Admin is disabled because insecure channel'))
According to https://groups.google.com/forum/#!searchin/web2py-developers/request.is_local/web2py-developers/kkBvSzX4wO8/Rjom8huf4yMJ , request.is_local is False behind the Apache proxy, so calling https://server/web2py/admin fails both request.is_https (since the proxy forwards to http://) and request.is_local. Commenting out this block causes login dialog to fail (for the same reasons). Thus the correct modification is to use request.is_local = True

2014-10-13

ETL for Reading ACL Analytics Exchange Server Job Logs (with AX Exception integration)

So you want have a machine way of reading the Job logs from ACL Analytics Exchange Server 3 or 4? Here's a SQL that lets you find out what analytic was started when and if any results were pushed to AX Exception: (Connect to the PostgreSQL database holding the backend of AX, by default it was called AclAuditExchangeDB in AX 3).
select t1.starttime, t2.name as analytic_name, 
t4.name as analytic_project, t6.name as activity, 
t7.name as engagement, t3.resulttable, t3.destinationentity, 
t3.destinationanalytic 
from scriptjobs t1
left outer join
audititems t2
on t1.analyticid = t2.id
left outer join
scriptjobpublish t3
on t1.jobnumber = t3.jobnumber
left outer join
audititems t4
on t2.parentid = t4.id
left outer join
audititems t5
on t4.parentid = t5.id
left outer join
audititems t6
on t5.parentid = t6.id
left outer join
audititems t7
on t6.parentid = t7.id

where t1.starttime > '2014-10-01'
order by engagement, activity, t1.starttime

2014-07-02

Stripping table name prefixes from SAP DirectLink fieldnames

So if you are using ACL DirectLink to do a full table extraction out of SAP, it will create an ACL Table whose fieldnames are prefixed with %TABLENAME%_, so for example BUKRS from BSAK ends up as BSAK_BUKRS. When you are using DirectLink as an ETL shim to export a flatfile extraction, you sometimes want the native field names instead, without prefixes (Careful: if you have specified a server side join in DirectLink, it will use the tablename prefix to distinguish between the same fieldname in two different tables from the resulting join, for example: BKPF_BUKRS, BSAK_BUKRS. Of course, server side joins are to be avoided when doing this type of dump anyway). From the ACL project, extract the table definition:
BSAK_MANDT     UNICODE     1   6   AS "Accounting: Secondary Index for Vendors (Cleared Items);Client"  
BSAK_BUKRS     UNICODE     7   8   AS "Accounting: Secondary Index for Vendors (Cleared Items);Company Code"  
BSAK_LIFNR     UNICODE    15  20   AS "Accounting: Secondary Index for Vendors (Cleared Items);Account Number of Vendor or Creditor"  
BSAK_UMSKS     UNICODE    35   2   AS "Accounting: Secondary Index for Vendors (Cleared Items);Special G/L Transaction Type"  
BSAK_UMSKZ     UNICODE    37   2   AS "Accounting: Secondary Index for Vendors (Cleared Items);Special G/L Indicator"  
BSAK_AUGDT     DATETIME   39  16   PICTURE "YYYYMMDD" AS "Accounting: Secondary Index for Vendors (Cleared Items);Clearing Date"  
BSAK_AUGBL     UNICODE    55  20   AS "Accounting: Secondary Index for Vendors (Cleared Items);Document Number of the Clearing Document"  
...
etc
$ awk '{print $1}' bsak_all.txt | while read line; do echo "DELETE FIELD " $(echo $line | sed 's/[^_]*_\(.*\)/\1/') " OK" && echo "DEFINE FIELD " $(echo $line | sed 's/[^_]*_\(.*\)/\1/') " COMPUTED " $line && echo; done

non-greedy matching in sed

If you want to match a prefix in a string and stop at the n'th occurence of a delimiter in that string then you need non-greedy matching. This is how you would do it if the regex engine you are using doesn't support it natively, do it by negative character class.

Example: given a URI http://service.domain.tld/path1/path2/path3 and you only want service.domain.tld by splitting on /, then:

$ echo "http://service.domain.tld/path1/path2/path3" | sed 's@http://\([^/]*\).*@\1@'
service.domain.tld
Now what happens if you need the suffix after the delimiter? Just change the capturing parens:
$ echo "foo_bar_baz" | sed 's/[^_]*_\(.*\)/\1/'
bar_baz

Courtesy of: http://stackoverflow.com/questions/1103149/non-greedy-regex-matching-in-sed

Find which SAP roles provide access to what T-Codes

A lot of times, your SAP superuser wants you to request the specific role (or a template user) for which to provision a new account or role assignment you requested. Best way to determine that role is to search roles which fulfill the TCODEs you require access to. To do this, make sure you can login with a user with a role that has access to SUIM.

SUIM -> Roles -> By Authorization Values

Set Authorization Object 1 to S_TCODE

Click Entry value

Specify the TCodes you want to look up roles for. Pay attention to the AND and OR boxes for multiple TCode criteria.